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FOREWORD 


The  Future  Battlefield  Conditions  (FBC)  team  of  the  Armored  Forces  Research  Unit 
(ARFU),  U.S.  Army  Research  Institute  for  the  Behavioral  and  Social  Sciences  has  a  Science  and 
Technology  Objective  (STO)  entitled  “Force  XX]  Training  Strategies.”  The  purpose  of  research 
under  this  STO  is  to  develop  and  demonstrate  prototype  training  and  evaluation  technologies  that 
prepare  operators  and  commanders  to  take  maximum  advantage  of  evolving  digital  systems. 

This  STO  is  also  reflected  in  the  FBC  work  package  (2228)  FASTRAIN:  Force  XXI  Training 
Methods  and  Strategies.  Design  of  prototype  commander  and  staff  training  packages  that  use 
advanced  digital  technology  was  completed  under  previous  work.  This  report  describes  work 
under  a  contract  entitled  “Performance  Evaluation,  Training,  and  Future  Requirements  for 
Digital  Skills.”  The  purpose  of  the  work  described  in  this  report  was  to  develop  and  evaluate 
performance  assessment  tools  for  future  commanders  and  staffs  working  in  a  future  digital 
environment. 

This  report  describes  the  design,  development,  and  demonstration  of  prototype  automated 
measures  designed  to  improve  training  and  evaluation  for  brigade  and  below  command  and  staff 
training.  The  report  examines  implementation  of  these  measures  in  a  Future  Combat  Command 
and  Control  (FCC^)  experiment  at  the  Mounted  Maneuver  Battlespace  Lab  located  at  Fort  Knox, 
Kentucky. 

At  least  tlrree  audiences  may  be  interested  in  this  report.  Materiel  and  training  developers 
could  consider  hardware  and  software  issues  involved  in  embedding  these  measures  into  future 
command,  control,  communications,  and  computer  systems.  Training  unit  and  training 
(simulation  site)  personnel  who  conduct  training  of  digital  staffs  could  consider  these  measures 
for  feedback  concerning  commander  and  staff  performance.  Also,  measurement  specialists  and 
researchers  could  examine  this  report  to  inform  further  research  into  the  development  of 
automated  measures  of  commander  and  staff  performance. 

In  addition  to  this  report,  products  developed  under  this  effort  include  software  to  run  an 
Observer  Workstation,  used  to  develop  automated  measures  of  commander  and  staff 
performance  during  FCC^.  Data  gathered  during  FCC^  is  also  contained  on  the  system. 

The  research  reflected  in  this  report  was  briefed  to  sponsors  throughout  the  effort  and  in  a 
final  in  progress  review,  held  at  AFRU,  Fort  Knox,  Kentucky,  on  25  July  2001. 

M.  SIMUTIS 
ical  Director 
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PROTOTYPE  AUTOMATED  MEASURES  OF  COMMAND  AND  STAFF  PERFORMANCE 


EXECUTIVE  SUMMARY 


Research  Requirement: 

This  research  and  development  effort  continues  the  work  by  the  U.S.  Army  Research 
Institute  for  the  Behavioral  and  Social  Sciences  (ARI),  Armored  Forces  Research  Unit,  Future 
Battlefield  Conditions  Team.  It  focuses  on  the  design  and  development  of  automated  training 
and  performance  evaluation  techniques.  A  primary  context  for  these  efforts  is  digital  brigade 
and  below  training  requirements  and  environments.  For  this  project,  ARI’s  objective  was  to 
design,  develop,  and  demonstrate  20  prototype  automated  measures  to  improve  training  and 
evaluation  for  brigade  and  below  command  and  staff  performance. 

The  prototype  automated  measures  developed  were  implemented  during  the  Future  Combat 
Command  and  Control  (FCC^)  Concept  Experimentation  Program  experiment  conducted  by  the 
Mounted  Maneuver  Battlespace  Lab  (MMBL)  at  Fort  Knox,  Kentucky.  The  ARFs  purpose  for 
participating  in  this  experiment  was  to  support  the  MMBL  and  the  Army’s  need  to  gain 
additional  information  on  future  staff  evaluation  requirements  in  a  virtual  simulation 
environment,  and  gather  feedback  for  improvements  to  the  prototype  automated  measures 
developed  during  this  effort. 

Procedure: 

The  Project  Team  reviewed  The  Standard  Army  After  Action  Review  System  (STAARS) 
handbook  (U.S.  Army  Training  and  Doctrine  Command,  1997)  and  previous  research  literature 
regarding  staff  processes  and  measures  developed  to  assess  them,  especially  automated 
measures.  This  literature  review  provided  the  basis  for  decisions  concerning  staff  processes  to 
measure,  as  well  as  guiding  the  procedures  that  the  Project  Team  would  use  to  design,  develop, 
implement,  and  analyze  the  measures.  A  total  of  29  candidate  automated  measures  were  then 
designed  and  presented  to  a  Subject  Matter  Expert  (SME)  Advisory  Group  (SAG)  for  review. 
The  SAG  selected  20  measures  for  development  by  the  Project  Team.  The  Project  Team  then 
built  a  prototype  observer  workstation  to  facilitate  the  use  of  the  automated  measures  by  trainers 
and  other  personnel  during  after  action  reviews  (AARs). 

The  prototype  automated  measures  were  implemented  during  the  FCC^  experiment,  which 
took  place  7  May  through  24  May  2001 .  The  experiment  was  conducted  in  the  MMBL  Mounted 
Warfare  Test  Bed  at  Fort  Knox  with  the  2"‘*  Squadron,  2"^  Armored  Cavalry  Regiment  from  Fort 
Polk  participating.  The  Project  Team  demonstrated  19  of  the  20  automated  measures  it  had 
designed  and  developed  using  commercial  business  software  in  common  use  throughout  the 
Army.  Development  of  one  measure  was  not  completed  during  the  project. 
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Findings: 


Based  on  the  FCC^  implementation,  additional  effort  is  required  to  complete  the  integration 
of  various  tools  associated  with  the  observer  workstation  so  that  raw  performance  data  can  be 
automatically  transformed  into  a  finished  AAR  product.  Additional  research  is  required  to 
establish  performance  standards  for  future  brigade  and  below  battle  staffs  using  advanced 
command,  control,  communications,  computers,  and  intelligence  (C^I)  systems  upon  which 
additional  automated  measures  can  be  developed.  Trained  SMEs  on  staff  operations  and 
procedures  are  still  required  as  observers  to  provide  a  context  for  the  results  obtained  by 
automated  performance  measures. 

This  research  effort  demonstrated  that  commercial  business  software  packages  operating  on 
low-cost  personal  computers  can  be  used  to  design,  develop,  and  implement  automated 
performance  measures  in  a  laboratory  environment.  With  additional  research,  they  may  be  able 
to  support  implementation  of  automated  performance  measurement  in  brigade-level  and  below 
battle  command  staffs  using  advanced  digital  O'*!  systems. 

Utilization  of  Findings: 

The  specific  audience  who  may  find  the  information  contained  in  this  report  beneficial 
includes:  (a)  measurement  specialists  and  researchers,  (b)  simulation  system  programmers 
(hardware  and  software),  (c)  training  unit  and  training  site  personnel,  and  (d)  personnel 
conducting  the  FCC^  experiment  AARs  and  preparing  the  FCC^  experimental  final  report. 
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PROTOTYPE  AUTOMATED  MEASURES 
OF  COMMAND  AND  STAFF  PERFORMANCE 


Introduction 

The  transition  to  the  Army’s  Objective  Force  is  characterized  by  challenges,  such  as  how  the 
Army  will  train,  maintain,  and  operate  as  an  information-age  force  (U.S.  Department  of  the 
Army  [DA],  2001).  The  foundation  of  the  future  Army  includes  an  increased  area  of  operation; 
increased  operational  tempo;  combining  branches  for  close  combat  with  fewer  systems;  non¬ 
linear,  asymmetrical  operations;  and  information  dominance  with  increased  situational 
awareness.  A  key  enabler  for  this  future  force  will  be  enhanced  commanders’  situational 
understanding,  which  will  flow  from  the  fielding  of  advanced  command,  control, 
communications,  computers,  and  intelligence  (C‘’l)  systems. 

Closely  linked  to  the  training  challenge  is  the  need  for  measurement,  both  to  allow  for 
feedback  and  performance  improvement  and  also  to  support  the  design  and  development  of  the 
training  programs.  Measures  of  performance  (MOPs)  focused  on  process,  and  measures  of 
effectiveness  (MOEs)  focused  on  outcome  are  critical  areas  of  interest.  However,  direct 
observation  and  objective  measurement  of  performance  during  training  are  difficult,  particularly 
for  command  and  control  (C^)  performance.  Subjective  methods  used  for  assessing 
performance  are  labor-intensive  approaches,  requiring  observers  with  high  subject  matter 
expertise.  Even  with  automated  data  collection  aids  such  as  electronic  clipboards  or  computer- 
assisted  observation  tools,  these  methods  are  inefficient.  They  are  also  subject  to  unreliability 
because  of  the  lack  of  standardization  among  observers.  Measurement  problems  are  further 
exacerbated  in  the  information-intensive  environment  of  digital  C^.  Observers,  like  users  and 
participants,  can  be  quickly  overwhelmed  with  the  amount  of  information  relevant  to 
performance. 

Digital  O'*!  systems,  however,  offer  an  exceptional  opportunity  for  more  efficient  and 
objective  methods  for  performance  measurement,  particularly  for  performance.  Digital  0^*1 
systems  have  organic  capabilities  that  should  allow  us  to  exploit  them  to  automatically  collect, 
analyze,  and  portray  data.  As  Goehring  (1995)  states: 

What  distinguishes  this  approach  [automated  analysis  by  computer  systems]  from 
previous  software  tool  development  efforts  is  that  most  of  the  work  of  the 
researcher  or  analyst  is  actually  accomplished  by  the  computer  software.  After 
careful  fonnulation  of  the  problem  and  codification,  the  actual  analysis  is 
accomplished  automatically.  The  difference  from  the  past  is  somewhat  subtle. 

Previously,  software  tools  processed  and  presented  refined  information  to  the 
investigator,  who  then  further  analyzed  the  information.  Now  because  of  several 
technological  developments  it  is  increasingly  possible  for  much  of  the  second 
phase  of  the  work  to  be  accomplished  fully  automatically.  In  the  first  case,  the 
computer  helped  to  do  the  work.  In  the  second  case,  the  computer  does  all  the 
work!  (pp.  4-5) 
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Integrating  commercial  business  software  into  digital  C^'l  systems  may  be  a  low-cost  way  to 
allow  trainers,  commanders,  and  observers  who  are  not  computer  programmers  to  design  their 
own  automated  performance  measures.  As  these  commercial  business  software  packages  are 
becoming  more  capable  and  sophisticated,  their  ease  of  use  and  relatively  low  cost  has  led  the 
Army  to  widely  distribute  them.  Increasingly,  every  soldier  is  being  provided  access  to  desktop 
computers  that  use  business  software  to  accomplish  routine  tasks.  These  soldiers  are  also 
experimenting  with  ways  to  use  this  software  to  collect,  process,  analyze,  and  disseminate 
information  about  performance. 

Capitalizing  on  this  broad-based  knowledge  and  familiarity  with  operating  commercial 
business  software  may  be  a  bridge  to  automating  performance  measurement  using  O'*!  systems. 
Although  outcome  information  (i.e.,  MOEs)  is  commonly  automated,  more  work  is  necessary  for 
process  information  (i.e.,  MOPs)  to  reach  the  same  level  (Salas,  Bowers,  &  Cannon-Bowers, 
1995).  The  U.S.  Army  Research  Institute  for  the  Behavioral  and  Social  Sciences  (ARI)  has 
sponsored  a  series  of  research  projects  with  the  objective  of  automating  performance 
measurement,  particularly  for  battle  staffs  equipped  with  advanced  digital  C^  systems.  This 
report  details  the  assessment  methodology  work  performed  for  an  ARI-sponsored  contract 
project  titled,  “Performance  Evaluation,  Training,  and  Future  Requirements  for  Digital  Skills” 
and  referred  to  herein  as  DC'^I-S.  The  research  effort  was  conducted  by  a  project  team  consisting 
of  personnel  from  the  Human  Resources  Research  Organization  and  Litton  PRC  (hereafter 
referred  to  collectively  as  the  Project  Team). 

Specifically,  the  objective  of  the  DC‘*I-3  research  project  was  threefold:  design,  develop, 
and  demonstrate  prototype  automated  measures  to  improve  training  and  evaluation  for  brigade 
and  below  command  and  staff  training;  design,  develop,  and  demonstrate  prototype  training 
techniques  that  incorporate  principles  of  cognitive  psychology  and  automated  performance 
assessment  and  feedback;  and  identify  research  issues  and  training  approaches  for  the  future 
force  with  a  focus  on  embedded  staff  training  and  multi-functional  leaders  and  soldiers.  This 
report  documents  the  design,  development  and  demonstration  of  the  prototype  automated 
measures  of  command  and  staff  performance.^  The  original  prototype  automated  measures 
package  was  designed  for  “Refinement  of  Methods  for  the  Training  and  Assessment  of  Digital 
Staffs  (DC'^I-2),”  and  is  described  in  the  report  Refinement  of Prototype  Staff  Evaluation 
Methods  for  Future  Forces:  A  Focus  on  Automated  Measures  (Throne,  Holden,  &  Lickteig, 
2000). 

There  are  many  benefits  to  using  automated  measures,  some  of  which  are  outlined  in  Throne 
et  al.  (2000).  Most  importantly,  as  command  staffs  rely  on  computers  in  their  work,  the  more 
important  it  will  become  to  assess  their  computer  interactions  as  meaningful  aspects  of  work 
process  and  outcome.  Second,  automated  measures  are  objective  measures  of  performance  - 
they  present  “unchallengeable  ‘ground  truths’  ”  (Brown  et  al.,  1997,  p.  3).  Third,  a  greater 
reliance  on  automated  measures  may  increase  the  scope  and  precision  of  performance  assessment 
and  feedback.  Fourth,  automated  measurement  and  analysis  may  be  needed  for  more  complex 
work  settings,  such  as  staff  performance.  Fifth,  unobtrusive  and  automatic  data  collection 
may  reduce  measurement  error  and  increase  the  accuracy  of  the  information  presented  during 


'  For  more  information  on  the  prototype  training  techniques,  see  Deatz  and  Campbell.  (2001).  For  more  information 
on  the  research  issues  for  the  future  force,  see  Campbell  and  Holden  (2001). 


2 


after  action  reviews  (AARs).  Finally,  automated  data  collection  will  reduce  observer  workload 
and  resource  requirements.  Current  AAR  presentations  can  be  very  labor-intensive  and  products 
are  often  requested  late  in  the  battle,  which  makes  them  difficult  to  prepare  in  time  for  the  AAR 
(Anderson,  Begley,  Amtz,  &  Meliza,  2000).  Automated  measurement  will  allow  these  products 
to  be  produced  in  a  more  timely  manner,  allowing  observers  to  focus  on  other  activities,  such  as 
evaluating  overall  performance  and  providing  coaching  and  mentoring  where  needed  (Morrison 
&  Meliza,  1999). 

Although  employing  automated  measures  is  beneficial,  they  should  not  be  the  only  form  of 
evaluation  used.  A  comprehensive  measurement  package  should  also  include  the  traditional 
types  of  performance  evaluation,  such  as  observations,  surveys,  and  interviews.  Automated 
measures  are  the  least  studied  aspect  of  a  well-rounded  measurement  package  and  should  only  be 
used  as  a  complement  to  the  other  types  of  measurement.  In  addition,  automated  measures 
provide  measures  of  what  happened,  not  why  it  happened.  In  order  to  provide  a  stronger  link 
between  outcomes  and  processes,  more  information  would  be  needed  than  that  provided  by 
automated  measures  alone.  This  project  focuses  solely  on  automated  measures  because  other 
aspects  of  measurement  are  more  fully  developed.  Additionally,  a  complementary  package  of 
traditional  measures  (e.g.,  survey,  observer,  and  interview  questions)  was  already  developed 
during  the  original  project  entitled  “Prototype  Methods  for  the  Design  and  Evaluation  of 
Training  and  Assessment  of  Digital  Staffs  and  Crewmen,”  and  referred  to  as  DC"*!  (Throne  et  al., 
1999). 


Organization  of  the  Report 
This  report  has  five  major  sections: 

•  Introduction.  Organization  of  this  report  as  well  as  summary  of  previous  research  and 
relevant  literature  on  team  performance  and  evaluation.  Includes  a  discussion  of  staff 
processes,  measures  of  staff  processes,  automated  measures  of  command  and  staff 
performance,  and  conclusions. 

•  Method.  Description  of  the  measures  development  process,  including  front-end  analysis, 
design,  and  development  of  automated  measures  of  command  and  staff  performance. 
Also  includes  a  description  of  the  Future  Combat  Command  and  Control  (FCC^^ 
participants,  materials,  and  implementation  of  the  developed  measures. 

•  Results  and  Discussion.  Representative  results  from  developed  measures  and  discussion 
of  automated  measures  evaluation  and  implementation  for  FCC^. 

•  Lessons  Learned:  Improve.  Summary  of  the  major  lessons  learned  concerning 
implementation  of  automated  measures  during  Mounted  Maneuver  Battlespace  Lab 
(MMBL)  experimentation. 

•  Lessons  Learned:  Sustain.  Principles  that  should  sustain  future  research  and 
development  efforts  on  the  use  of  automated  measures  to  assess  command  and  staff 
performance. 
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Appendix  A  contains  a  list  of  the  acronyms  and  abbreviations  used  in  this  report.  Appendix 
B  contains  a  description  of  the  FCC^  setting.  Appendix  C  contains  the  key  aspects  of  the 
prototype  automated  measurement  package,  including  operational  definitions,  rationale,  and 
recommended  output  formats  for  the  candidate  automated  measures.  Appendix  D  contains  a 
sample  automated  measure  macro. 

Background 

The  purpose  of  the  Background  is  to  provide  the  context  for  the  DC^'l-S  project  and  the 
rationale  for  the  automated  measures  approach  to  evaluation.  A  brief  summary  of  the  previous 
work  conducted  for  the  DC'*!  and  DC'’l-2  projects  and  the  lessons  learned  from  each  are 
provided.  The  environment  in  which  the  DC'’l-3  research  was  implemented  is  also  described. 

For  a  review  of  additional  research  related  to  automated  measures  of  command  and  staff 
performance,  see  Throne  et  al.  (2000). 

dC I  Research 

The  development  of  automated  measures  of  performance  was  only  a  small  portion  of  the 
original  DC"*!  project.  The  goal  of  the  project  was  to  develop  a  prototype  staff  training  and 

assessment  package.  For  the  assessment  portion,  the  Project  Team  developed  not  only 
automated  measures,  but  also  surveys,  interview  questions,  and  an  Observer  Data  Collection 
Instrument  (Throne  et  ah,  1999). 

In  order  to  develop  effeetive  staff  performance  measures,  the  term  “staff’  had  to  be 
defined.  Since  information  about  automated  measures  of  command  and  staff  performance  is 
limited,  the  DC‘*I  Project  Team  redefined  staff  performance  as  team  performance,  since  a 
military  staff  could  be  viewed  as  a  team.  Therefore,  the  definition  of  teams  developed  by  Salas, 
Dickinson,  Converse,  and  Taimenbaum  (1992)  was  used  to  refer  to  a  staff  They  define  team  as 
“a  distinguishable  set  of  two  or  more  people  who  interact  dynamically,  interdependently,  and 
adaptively  toward  a  common  and  valued  goal/objective/mission,  who  have  each  been  assigned 
specific  roles  or  functions  to  perform,  and  who  have  a  limited  life-span  of  membership”  (p.  4). 

Since  the  measurement  package  was  to  be  implemented  during  the  Battle  Command 
Reengineering  (BCR)  III,  the  Battle  Lab  Experiment  Plan  (BLEP)  for  the  BCR  III  (MMBL, 

1998)  was  examined  as  a  starting  point  for  evaluation.  In  the  BLEP,  several  research  issues  or 
questions  related  to  advanced  digitization’s  effects  on  battle  command  at  brigade  and  below  were 
identified  that  could  form  the  basis  for  developing  future  staff  training  performance  standards. 
Since  the  future  battalion  battle  staff  model  that  the  Project  Team  was  using  did  not  have 
published  doctrine  or  tactics,  techniques,  and  procedures  upon  which  to  base  training  and 
performance  evaluations,  the  MMBL  issues  were  used  as  a  starting  point. 

A  total  of  14  automated  measures  were  designed  to  support  six  issues  that  the  Project  Team 
felt  could  be  partially  addressed  by  automated  processing  of  BCR  III  data.  The  intent  was  to  use 
the  data  collection  and  analysis  capabilities  of  the  MMBL  to  automatically  produce  the  output 
formats,  which  consisted  of  tables  and  graphs.  This  approach  did  not  work  and  to  produce  the 
results  for  the  issues,  a  considerable  amount  of  additional  data  examination,  editing,  processing. 
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and  subsequent  analyses  also  had  to  be  accomplished  by  the  DC"*!  Project  Team.  The  measure 
tables  and  graphs  were  manually  created  using  Microsoft®  Excel  with  no  automated  data 
processing  involved. 

The  most  important  lesson  learned  from  the  DC"^!  research  was  that  by  specifying  the  output 
format  of  the  automated  measures,  the  researcher  wilt  greatly  reduce  the  programmer’s  work  in 
extracting  data  from  the  system.  Output  format  specification  will  also  provide  the 
programmer  a  greater  understanding  of  what  the  researcher  is  looking  for  in  the  measures.  By 
providing  the  programmer  both  the  operational  definition  of  an  automated  measure  and  the 
specific  format  in  which  it  is  to  be  reported,  the  programmer  will  be  better  able  to  meet  the 
expectations  of  the  researcher  (Throne  et  al.,  1999). 

DC*  1-2  Research 

For  the  DC'^I-2  project,  automated  measures  of  command  and  staff  performance  became  the 
focus.  An  opportunity  to  implement  these  measures  was  provided  by  BCR  IV,  which  took  place 
in  April,  2000.  By  participating  in  the  BCR  IV,  the  Project  Team  had  the  opportunity  to  conduct 
a  trial  implementation  of  the  automated  measures  of  performance.  Coordination  between  ARI 
and  the  MMBL  at  Fort  Knox,  Kentucky,  enabled  the  two  organizations  to  work  together  as  a 
team  to  accomplish  multiple  goals.  The  Project  Team  used  the  preliminary  automated  measures 
developed  for  the  DC^^I  project  as  a  starting  point.  For  this  second  project,  since  there  already 
was  a  clear  understanding  of  what  constitutes  a  team,  the  next  step  was  to  establish  aspects  of 
positive  team  performance  in  order  to  determine  what  to  measure.  Consequently,  the  Project 
Team  decided  to  institute  a  framework  around  which  to  develop  the  measures.  The  Project  Team 
needed  to  determine  what  kind  of  processes  to  measure  that  would  provide  an  indication  of  level 
of  staff  performance. 

After  a  literature  review,  the  Project  Team  found  that  one  of  the  most  thorough  evaluations 
of  team  processes  was  conducted  by  Cannon-Bowers,  Tannenbaum,  Salas,  and  Volpe  (1995). 
These  authors  conclude  that  there  are  eight  skill  dimensions  common  to  most  team-based  tasks. 
The  definitions  given  by  Cannon-Bowers  et  al.  are  provided  in  Table  1 .  As  corroboration,  the 
U.S.  Army  Training  and  Doctrine  Command  (TRADOC)  Regulation  350-70  (DA,  1999)  lists 
team  skill  requirements  very  similar  to  those  outlined  by  Cannon-Bowers  et  al.  Therefore,  the 
Cannon-Bowers  et  al. /TRADOC  framework  of  team  process  skill  dimensions  was  implemented. 
However,  after  careful  deliberation,  only  six  of  the  eight  skill  dimensions  were  selected  as 
potential  candidates  for  being  partially  addressed  by  automated  measures.  The  two  dimensions 
not  selected,  Leadership/Team  Management  and  Interpersonal  Relations,  were  considered  either 
to  be  more  of  an  individual  skill  or  to  require  observer  input  in  order  to  be  measured. 
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Table  1 


Cannon-Bowers  et  al.’s  Team  Process  Skill  Dimension  Definitions 


Team  Process  Skill  Dimension 

Definition 

Adaptability 

The  process  by  which  a  team  is  able  to  use  information  gathered 
from  the  task  environment  to  adjust  strategies  through  the  use  of 
compensatory  behavior  and  reallocation  of  intrateam  resources. 

Perfonnance  Monitoring  and 
Feedback 

The  ability  of  team  members  to  give,  seek,  and  receive  task- 
clarifying  feedback;  includes  the  ability  to  accurately  monitor  the 
performance  of  teammates,  provide  constructive  feedback 
regarding  errors,  and  offer  advice  for  improving  performance. 

Shared  Situational  Awareness 

The  process  by  which  team  members  develop  compatible  models 
of  the  team’s  internal  and  external  environment;  includes  skill  in 
arriving  at  a  common  understanding  of  the  situation  and  applying 
appropriate  task  strategies. 

Leadership/Team  Management 

The  ability  to  direct  and  coordinate  the  activities  of  other  team 
members,  assess  team  performance,  assign  tasks,  motivate  team 
members,  plan  and  organize,  and  establish  a  positive  atmosphere. 

Interpersonal  Relations 

The  ability  to  optimize  the  quality  of  team  members’  interactions 
through  resolution  of  dissent,  utilization  of  cooperative  behaviors, 
or  use  of  motivational  reinforcing  statements. 

Communication 

The  process  by  which  information  is  clearly  and  accurately 
exchanged  between  two  or  more  team  members  in  the  prescribed 
manner  and  with  proper  terminology;  the  ability  to  clarify  or 
acknowledge  the  receipt  of  information. 

Coordination 

The  process  by  which  team  resources,  activities,  and  responses  are 
organized  to  ensure  that  tasks  are  integrated,  synchronized,  and 
completed  within  established  temporal  constraints. 

Decision-Making 

The  ability  to  gather  and  integrate  information,  use  sound 
judgment,  identify  alternatives,  select  the  best  solution,  and 
evaluate  the  consequences. 

Note.  Adapted  from  Cannon-Bowers  et  al.  (1995),  pp.  344-346. 


Once  the  framework  was  selected  and  the  processes  to  be  measured  were  defined,  the 
Project  Team  needed  to  define  the  phrase  “automated  measures.”  However,  very  few 
researchers  had  developed  automated  measures  and  most  of  those  who  had  were  measuring 
either  individual  performance  or  outcomes  of  team  performance,  not  processes.  Those  few  who 
had  measured  team  performance  automatically  (e.g.,  Atwood  et  al.,  1991;  Leibrecht,  Meade, 
Schmidt,  Doherty,  &  Lickteig,  1 994)  had  not  provided  a  specific  definition  of  what  constitutes 
an  automated  measure  of  team  performance.  Therefore,  based  on  the  literature  reviewed,  the 
Project  Team  developed  its  own  definition.  Automated  measures  were  defined  as  measures 
based  on  data  collected  from  a  C”*!  system  and  processed  automatically  by  preformatted  routines 
to  provide  meaningful  training  and  performance  assessment  feedback  with  as  little  observer  input 
as  possible.  The  C'^I  system  data  was  further  defined  to  include,  but  not  limited  to,  voice 
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communications,  simulation  protocol  data  units  (PDUs),  electronic  messaging,  system  usage, 
situational  awareness  information,  and  observer  input. 

The  first  step  in  the  design  process  was  to  select  candidate  measures  for  development.  For 
each  of  the  six  team  process  skill  dimensions  chosen,  multiple  candidate  measures  were 
identified  in  order  to  obtain  supporting  data  on  that  particular  skill  dimension.  Candidate 
measures  were  selected  from  those  implemented  during  the  BCR  III,  measures  developed  by 
other  researchers  (e.g.,  Dzindolet  et  al.,  1998;  Mason,  1995),  or  based  on  the  lessons  learned 
from  BCR  III  discussed  in  Throne  et  al.  (1999).  All  candidate  measures  were  designed  so  their 
outputs  could  be  displayed  in  at  least  one  of  these  three  formats:  tables,  graphs,  and  pictures, 
with  an  emphasis  placed  on  pictures. 

Once  the  candidate  measures  were  designed,  they  were  passed  to  the  Subject  Matter  Expert 
(SME)  Advisory  Group  (SAG)  for  review  and  approval.  The  SAG  was  a  committee  consisting 
of  participants  from  the  MMBL,  the  MMBL  Core  Support  Group,  ARI,  and  the  Project  Team. 
This  group  was  organized  during  DC'‘l-2  in  an  attempt  to  include  programmers,  SMEs,  and 
researchers  in  the  entire  measures  design  and  development  process.  Based  on  the  SAG’s 
feedback,  the  Project  Team  met  with  MMBL  system  programmers  to  begin  developing  the 
higher  priority  measures.  After  meeting  with  the  programmers,  some  of  the  priority  measures 
were  considered  to  be  too  time-consuming  or  expensive  to  develop,  given  the  BCR  IV  time  and 
cost  constraints.  Of  the  30  candidate  measures,  19  measures  were  selected  as  feasible  for 
development,  given  the  BCR-related  costs  and  time  constraints.  These  measures  are  summarized 
in  Table  2. 

Table  2 


Automated  Measures  Developed  for  Battle  Command  Reengineering  IV  by  Skill  Dimensions 


Skill  Dimension  and  Measure 
Name 

Description 

Adaptability 

Terrain  Analysis 

Amount  of  time  each  duty  position  uses  each  of  the  following  tools: 
Stealth  Control,  Terrain  Intervisibility,  FOV,  Snail  Display,  and  PLOT 
Display  during  the  mission. 

Performance  Monitoring  and  Feedback 

SITREP  Use  Frequency  and  duration  of  use  of  the  SITREP  tool  during  each  mission 

by  duty  position. 

SPOTREP  Use 

Frequency  and  duration  of  use  of  the  SPOTREP  tool  during  each 
mission  by  duty  position. 

CCIR 

Frequency  and  duration  of  use  of  the  CCIR  tool  during  each  mission 
by  duty  position. 

Information  Retrieval  by 
the  Commander 

Type  and  frequency  of  information  the  Commander  retrieves  on  his 
own  that  other  staff  normally  retrieve  for  him. 

(table  continues) 


1 


Table  2  (Continued) 


Skill  Dimension  and  Measure 
Name 

Description 

Shared  Situational  Awareness 

Map  Area 

Square  kilometers  of  the  battlefield  displayed  and  center  point  of  each 
resized  PVD  map  during  each  mission  by  duty  position  at  critical 
points  in  time. 

Line  of  Sight 

Frequency  and  duration  of  use  of  the  Line  of  Sight  tool  during  each 
mission  by  duty  position. 

Surprise  Attack 

Total  number  of  flank  or  rear  engagements  on  OPFOR  and  BLUFOR 
vehicles  during  each  mission. 

Collateral  Damage 

Total  number  of  attacks  on  BLUFOR  non-instrumented  vehicles 

and/or  personnel  by  indirect  non-line  of  sight  weapon  systems  under 
battalion  control  during  each  mission. 

Communication 

Whiteboard  Use 

Total  number  of  Whiteboard  files  residing  on  each  workstation  for 
each  mission  by  duty  position. 

Radio  Communications 
Pattern 

Frequency  and  duration  of  use  of  battalion  command  and  operations- 
intelligence  radio  nets  and  Whiteboard  conferencing  during  each 
mission  by  duty  position  at  critical  points  in  time. 

Personnel  Initiating 

Total  number  of  Whiteboard  conferences,  lasting  3  minutes  or  more, 

Whiteboard  Conferences 

initiated  during  each  mission  by  duty  position. 

Coordination 

Overlay  Use 

Total  number  of  workstations  showing  the  same  operations  overlay 
file  that  the  Squadron  Commander  is  showing  on  his  PVD  for  each 
mission  by  duty  position  at  critical  points  in  time. 

Whiteboard  Commonality 

Total  number  of  Whiteboard  directories  showing  the  same  Whiteboard 
files  that  the  Commander  and  Deputy  Commander  have  for  each 
mission  by  duty  position  at  critical  points  in  time. 

Targeting 

Total  number  of  times  a  SPOTREP  query  was  conducted  on  the  target 
identified  in  a  fire  support  request  immediately  before  it  was 
transmitted  by  a  staff  member. 

Fire  Support 

Ratio  of  OPFOR  kills  due  to  indirect  fire  from  units  controlled  by  the 

Coordination 

squadron  staff  to  OPFOR  kills  due  to  direct  fire  controlled  by 
squadron  subordinate  units. 

Fire  Engagements 

Average  range  of  OPFOR  and  BLUFOR  kills  by  vehicle  type  during 
each  mission. 

OPFOR  Destruction 

Time  from  the  first  OPFOR  engagement  until  OPFOR  vehicle  losses 
exceeded  70%.  In  addition,  the  rate  at  which  the  OPFOR  was  killed  in 
5-minute  intervals  from  the  first  engagement  until  the  last  OPFOR  kill 
during  each  mission. 

(table  continues) 
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Table  2  (Continued) 


Skill  Dimension  and  Measure 
Name 

Description 

Decis  ion-Making 

UAV  Effectiveness 

Percentage  of  OPFOR  vehicles  first  detected  by  the  UAV  under 
squadron  control  for  each  UAV  launch  during  each  mission. 

Note.  BLUFOR  =  blue  forces;  CCIR  =  commander’s  critical  information  requirements;  PLOT  = 
forward  line  of  own  troops;  FOV  =  field  of  view;  OPFOR  =  opposing  forces;  PVD  =  plan  view 
display;  SITREP  =  situation  report;  SPOTREP  =  spot  report;  UAV  =  unmanned  aerial  vehicle. 
From  Refinement  of prototype  staff  evaluation  methods  for  future  forces:  A  focus  on  automated 
measures,  by  M.  H.  Throne,  W.  T.  Holden,  Jr.,  and  C.  W.  Lickteig,  2000,  Alexandria,  VA:  U.S. 
Army  Research  Institute  for  the  Behavioral  and  Social  Sciences. 

Unfortunately,  since  the  pictorial  and  graphical  representations  for  a  majority  of  the 
automated  measures  could  not  be  supported  by  the  MMBL’s  programming  analysts  due  to  time 
constraints,  the  automated  measures  outputs  provided  were  in  tabular  form  (Throne  et  al.,  2000). 
However,  the  Project  Team  attempted  to  develop  prototype  pictorial  formats  for  some  of  the 
measures  based  on  the  data  tables.  These  picture  formats  were  all  developed  by  the  Project 
Team  using  a  commercial  off-the-shelf  spreadsheet  software  program,  Microsoft®  Excel.  This 
was  a  time-consuming  process  since  iterative  experimentation  with  the  picture  formats  was 
required  to  relate  data  tables  to  the  operational  context. 

In  summary,  although  a  prototype  automated  measurement  package  of  team  process  skills 
was  developed  for  DC'*I-2,  true  implementation  of  automated  measures  as  defined  by  the  Project 
Team  was  not  achieved.  Nevertheless,  considerable  progress  was  made  in  matching  Data 
Collection  and  Analysis  System  (DCA)  output  data  to  team  process  skill  dimensions  and  a  start 
was  made  on  converting  the  measures  data  into  automated  pictorial  representations  of  staff 
performance  (Throne  et  al.,  2000). 


Current  Project 

For  the  current  project,  the  Project  Team  was  directed  to  design  25  prototype  automated 
measures  and  then  select  20  for  development  and  implementation.  These  prototype  measures 
could  either  be  new  measures  or  measures  refined  from  the  DC'^I-2  project,  since  the  measures 
from  that  project  were  not  truly  automated.  The  goal  was  to  produce  as  many  measures  in 
pictorial  format  as  possible,  since  pictures  are  a  good  way  to  present  complex  measures  of 
command  and  staff  performance  in  a  manner  that  is  simple  to  interpret  (Meliza,  Bessemer, 
Burnside,  &  Shlechter,  1 992).  As  mentioned  previously,  although  this  report  focuses  on  the  use 
of  automated  measures,  they  should  be  used  as  one  aspect  of  a  complete  measurement  package. 

An  opportunity  was  provided  to  implement  the  prototype  automated  measurement  package 
during  the  MMBL’s  Concept  Experimentation  Plan,  FCC^,  previously  known  as  BCR.  A 
description  of  the  FCC^’  including  experimental  objectives,  is  provided  at  Appendix  B.  The 
Project  Team  had  extensive  experience  operating  in  this  environment  and  felt  that  there  would  be 
significant  advantages  to  eontinue  to  work  with  the  MMBL.  Among  these  are: 
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•  Army  Battalion  Staff  Participation.  An  intact,  trained  staff  from  a  maneuver  battalion 
would  be  participating  in  FCC^.  This  staff  would  replicate  the  rank  strueture  and 
operational  experience  of  soldiers  on  future  battle  staffs. 

•  Surrogate  Command,  Control,  Communications,  and  Computers  (SC'*)  System.  Staff 
members  in  FCC^  would  be  equipped  with  individual  SC'*  systems  that  are  digitally 
networked  to  share  information.  Eaeh  SC'*  workstation  is  equipped  with:  an  electronic 
map  that  displays  automatic  updates  of  friendly  and  opposing  forces’  (OPFOR’s)  location 
and  status;  e-mail  messaging;  collaborative  mission  planning  tools,  including  electronic 
Whiteboard  conferencing;  voice  communications;  and  a  computer-generated  virtual 
display  of  the  battlefield,  to  include  terrain  information  and  both  friendly  and  OPFOR 
vehicles  and  personnel.  Conceptual  designs  for  future  Army  C'*I  systems  include  most  of 
these  capabilities.  A  complete  description  of  the  SC'*  system  can  be  found  in  Throne  et 
al.  (2000). 

•  Instrumentation  and  Data  Collection  System.  Data  to  create  the  automated  measures 
could  be  generated  by  the  MMBL’s  DCA  during  the  virtual  simulation  used  in  the  FCC^. 
The  DCA  is  a  set  of  tools  designed  to  collect,  reduce,  and  display  information  on 
battlefield  performance,  command  and  control,  communications,  and  other  types  of  data 
in  distributed  simulations.  Again,  based  on  prior  efforts,  the  Projeet  Team  had  extensive 
knowledge  of  the  type  of  information  the  DCA  could  produce,  and  what  its  strengths  and 
limitations  were. 

•  Experimental  Objectives.  The  FCC^  experiment  was  designed  to  examine  the  effects  of 
digitization  on  battle  command  at  brigade  and  below.  This  supported  the  DC'*I-3 
project’s  main  objective,  which  was  to  design,  develop,  and  demonstrate  prototype 
automated  measures  to  improve  training  and  evaluation  for  future  brigade  and  below 
command  and  staff  training 


Method 

This  section  provides  both  the  measures  development  and  the  measures  implementation 
processes.  The  measures  development  process  ineludes  information  on  the  front-end  analysis, 
measures  design,  and  actual  measures  development.  The  measures  implementation  process 
includes  information  on  how  the  measures  were  implemented  during  the  FCC^. 

Measures  Development  Process 

The  measures  development  process  is  an  iterative  procedure  that  requires  extensive 
eollaboration  among  researchers,  SMEs,  and  programmers.  The  process  implemented  by  the 
Projeet  Team  consisted  of  12  basic  steps: 

1 .  Review  lessons  learned  from  previous  work 

2.  Identify  candidate  measures 

3.  Select  25  for  potential  development 


10 


4.  Draft  definition  and  output  format 

5.  Categorize  by  team  process  skill  dimension 

6.  Meet  with  the  SAG  to  select  measures 

7.  Specify  definition  and  output  format  for  selected  measures 

8.  Develop  software  code  and  run  on  previous  BCR  data  files 

9.  Assess  input  and  output 

10.  Repeat  steps  7  through  9,  as  required 

1 1 .  Develop  Observer  Work  Station 

12.  Add  measures  and  outputs  to  DC  A  Library 

The  front-end  analysis  stage  consisted  of  steps  1  and  2;  the  measures  design  stage  consisted 
of  steps  3  through  7;  and  the  measures  development  stage  consisted  of  steps  8  through  12.  The 
steps  are  discussed  by  the  stages  in  more  detail  below. 

Front-End  Analysis 

The  front-end  analysis  of  the  measures  development  began  with  a  look  at  the  lessons  learned 
from  the  previous  project  (Throne  et  al,  2000).  These  lessons  learned  provided  direction  for  the 
refinement  of  the  automated  measures  implemented  in  the  previous  project  as  well  as  ideas  for 
new  measures  that  might  provide  meaningful  feedback.  A  brief  summary  of  the  lessons  learned 
is  provided  below. 

One  lesson  learned  was  that  an  iterative  process  for  designing,  developing,  testing,  and 
refining  automated  measures  is  required  to  get  useful  results.  Programmers,  SMEs,  and 
researchers  need  to  follow  up  on  all  measure  outputs  to  make  certain  the  end  product  meets 
everyone’s  expectations.  For  the  current  project,  iterative  collaboration  was  more  fully 
integrated  into  the  design  and  development  process. 

Another  important  point  is  that  data  reduction  is  a  time-consuming  process.  During  the 
design  and  development  process,  the  Project  Team  had  projected  that  various  summary  tables 
would  provide  the  desired  results  with  little  additional  processing.  Flowever,  in  analyzing  the 
initial  results,  the  Project  Team  needed  to  refilter  the  data  tables  or  go  back  to  the  raw  data  to  get 
the  desired  measure  output.  There  is  a  trade-off  between  getting  the  most  meaningful  measures 
output  and  having  a  fully  automated  measures  development  system.  In  a  fully  automated 
measures  development  system,  data  reduction  would  not  be  as  time-consuming,  yet  the  data  used 
may  not  always  provide  the  most  meaningful  output.  One  of  the  main  purposes  of  this  project 
was  to  more  fully  automate  the  measures  development  and  output  process. 

After  incorporating  the  lessons  learned  into  the  measures  development  process,  the  Project 
Team  identified  potential  measures  for  development.  The  team  looked  for  measures  that  could 
be  automated  and  focused  heavily  on  measures  that  could  be  delivered  in  a  pictorial  format.  The 
ARI  requirement  was  to  design  25  automated  measures,  with  at  least  4  from  the  measures 
outlined  in  the  Standard  Army  After  Action  Review  System  (STAARS)  After  Action  Review  (AAR) 
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Handbook  (TRADOC,  1997)  and  at  least  1 1  from  the  measures  developed  during  the  previous 
project.  Therefore,  the  team  reviewed  the  TRADOC  handbook  and  the  work  done  on  the  DC'*I-2 
project. 

Measures  Design 

Measures  were  reviewed  and  selected  based  on  whether  they  were  relevant  to  the  BCR 
environment,  could  be  automated  without  the  need  of  an  observer,  would  be  of  interest  to  a 
commander,  and  could  possibly  be  depicted  in  a  pictorial  format.  From  the  STAARS  handbook 
(TRADOC,  1997),  12  potential  measures  were  chosen,  including  4  from  the  C^  battlefield 
operating  system  (BOS).  An  additional  8  potential  measures  were  chosen  from  the  DC'^I-2 
project.  Finally,  9  measures  were  chosen  that  appeared  in  both  the  STAARS  handbook  as  well 
as  in  the  DC'*I-2  project.  These  9  measures  were  essentially  measuring  the  same  things  although 
they  may  have  had  different  names  and/or  output  formats.  For  example,  the  STAARS  handbook 
contains  a  measure  named  fratricide,  which  was  called  collateral  damage  in  the  DC‘*I-2  project. 

In  summary,  there  were  12  unique  STAARS  measures,  9  STAARS/DC'^I-2  measures,  and 
8  unique  DC'‘l-2  measures,  for  a  total  of  29  potential  measures. 

Once  these  29  measures  were  selected,  operational  definitions,  rationale,  and  sample  output 
formats  were  produced  for  each.  Operational  definitions  and  rationale  were  adapted  from  the 
STAARS  handbook  (TRADOC,  1997)  as  well  as  from  those  developed  for  the  DC'^I-2  project. 
Measures  that  had  been  implemented  during  the  BCR  IV  were  evaluated  for  their  output  formats. 
The  original  outputs  provided  in  the  STAARS  handbook  were  also  evaluated  for 
meaningfulness.  Those  that  were  determined  to  be  meaningful  and  could  not  be  improved  were 
used  for  the  current  project  as  sample  output  formats.  Measures  that  did  not  provide  very 
meaningful  outputs  were  redesigned  to  provide  the  desired  information  in  the  best  possible 
format. 

The  next  step  was  to  integrate  the  STAARS  measures  with  the  DC‘*I-2  measures  and 
categorize  them  under  a  single  framework.  Whereas  the  STAARS  measures  are  classified  by 
BOS,  the  DC'^I-2  measures  were  classified  by  the  team  process  skill  dimensions  discussed 
earlier.  The  Project  Team  again  opted  to  use  the  Cannon-Bowers  et  al.  (1995)/TRADOC  (DA, 
1999)  framework  since  it  had  provided  a  meaningful  base  on  which  to  build  the  measures  for  the 
previous  project.  Through  discussions  among  researchers  and  SMEs,  measures  were 
individually  categorized  by  the  skill  dimensions.  Although  some  of  the  measures  may  have  fit 
under  more  than  one  skill  dimension,  they  were  categorized  according  to  the  skill  dimension  for 
which  they  were  most  relevant. 

The  operational  definitions,  rationale,  and  sample  output  formats  were  then  passed  along  to 
ARI  for  evaluation  and  approval.  After  evaluating  the  measures,  ARI  deleted  several  of  the 
measures  that  dealt  with  tool  use,  added  a  few  new  measures,  and  refined  the  output  formats  for 
some  of  the  remaining  measures.  This  review  process  led  to  24  proposed  measures.  Figure  1 
shows  the  24  candidate  measures  presented  under  the  team  process  skill  dimensions  framework. 
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Adaptability 

Decision-Making 

Node  Locations  (D) 

Sensor  Coverage  (D/S) 

CSS  Locations  (S) 

Multiple  Fire  Engagements  (D) 

Communication 

Maneuver  Battle  Sets  (S) 

Orders  Distribution  (D/S) 

CSS  Available  Over  Time  (S) 

Coordination 

Performance  Monitoring  and  Feedback 

Damage  to  BLUFOR/OPFOR  Systems  (D/S) 

Priority  Intelligence  Requirements  (D) 

Degradation  of  Forces  (D/S) 

Common  Map  Display  (D) 

Subordinate  Unit  Graphics  (S) 

Effects  of  Targeting  (S) 

Overlay  Use  (D) 

Shared  Situational  Awareness 

Counterreconnaissance  Effectiveness  (S) 

Map  Area  (D) 

Artillery  and  Counterfire  Radar  Coverage  (S) 

Surprise  Attack  (D) 

Kill  Distance  (D) 

Fire  Support  Coordination  (D/S) 

Sensor-Shooter  Time  Lag  (D) 

Air  Defense  Coverage  (S) 

Fratricide/Collateral  Damage  (D/S) 

Common  Picture  (D) 

Note.  D  =  DC'*I-2  measure;  S  =  STAARS  measure;  D/S  =  DC‘*I-2  and  STAARS  measure. 


Figure  1.  Proposed  measures  categorized  by  team  process  skill  dimensions. 

These  refined  measures  were  then  given  to  the  SAG  for  evaluation.  After  reviewing  the 
measures,  the  SAG  met  and  provided  ratings  on  which  measures  should  be  developed  now, 
developed  later,  or  not  developed.  The  Project  Team  consolidated  the  ratings  and  selected  the 
top  20  measures,  which  all  fell  into  the  first  two  categories,  for  development.  Table  3  shows  the 
top  20  measures  selected  for  development.  The  operational  definitions,  rationale,  and 
recommended  output  formats  for  these  measures  can  be  seen  in  Appendix  C. 

Table  3 


Selected  Measures  Categorized  by  Team  Process  Skill  Dimensions 


Skill  Dimension  and 
Measure  Name 

Description 

Adaptability 

Node  Locations 

Location  of  each  node  in  relation  to  major  subordinate  units  at  a  specified 
time  period. 

CSS  Locations 

Location  of  types  of  CSS  assets  at  a  specified  time  period. 

Communication 

Overlay  Use 

Compares  Commander’s  operation  overlay  files  usage  with  key  unit 
personnel  during  each  mission. 

(table  continues) 
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Table  3  (Continued) 


Skill  Dimension  and 
Measure  Name 

Description 

Coordination 

Damage  to  BLUFOR/ 

Number  and  type  of  BLUFOR/OPFOR  systems  out  of  action  and  what 

OPFOR  Systems 

damaged  or  destroyed  them  during  each  mission. 

Degradation  of  Forces 

Depiction  of  the  relative  combat  power  of  maneuver  forces  over  time  for 
both  BLUFOR  and  OPFOR  during  each  mission. 

Counterreconnaissance 

Effectiveness  of  the  BLUFOR  counterreconnaissance  effort  during  each 

Effectiveness 

mission. 

Artillery  and 
Counterfire  Radar 
Coverage 

Range  of  BLUFOR  artillery  (by  unit  or  type)  coverage  at  a  specified  time 
period. 

Kill  Distance 

Distance  between  OPFOR/BLUFOR  weapon  system  killed  and 
BLUFOR/OPFOR  system  that  killed  it  during  each  mission. 

Sensor- Shooter  Time 

Time  between  first  detection  of  OPFOR  HVT/HPT  by  BLUFOR  and  when 

Lag 

the  OPFOR  HVT/HPT  was  killed  during  each  mission. 

Decision-Making 

Sensor  Coverage 

Range  of  BLUFOR  artillery  (by  unit  or  type)  coverage  at  a  specified  time 
period. 

Multiple  Fire 
Engagements 

Number  of  OPFOR  vehicles  which  were  engaged  multiple  times  during 
each  mission. 

Maneuver  Battle  Sets 

Disposition  of  OPFOR  and  BLUFOR  at  a  specified  time  period. 

CSS  Available  Over 
Time 

Availability  of  ammunition  and  fuel  during  each  mission. 

Performance  Monitoring  and  Feedback 

Effects  of  Targeting 

Depiction  of  HVT/HPT  and  the  degree  of  attrition  each  suffered  during 
each  mission. 

Shared  Situational  Awareness 

Map  Area 

Square  kilometers  of  battlefield  displayed  for  each  staff  member  at  a 
specified  time  period. 

Surprise  Attack 

Depiction  of  BLUFOR  in  relation  to  OPFOR  when  BLUFOR  were 
attacked  for  flank  or  rear  engagements  during  each  mission. 

Fire  Support 
Coordination 

Total  number  of  artillery  and/or  mortar  rounds  fired  during  each  mission. 

Air  Defense  Coverage 

Depiction  of  air  defense  system  range  templates  overlayed  on  the  location 
of  the  unit’s  critical  assets  at  a  specified  time  period. 

Fratricide/Collateral 

Depiction  of  the  location,  unit(s)  involved,  and  results  of  fratricide  and 

Damage 

collateral  damage  during  each  mission. 

Common  Picture 

Difference  between  electronic  map  displays  at  a  specified  time  period. 

Note.  CSS  =  combat  service  support;  BLUFOR  =  blue  forces;  OPFOR  =  opposing  forces; 
HVT  =  high  value  target;  HPT  =  high  priority  target 
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Measures  Development 

The  Project  Team’s  goal  was  to  develop  fully  automated  measures  of  performance  using 
commercial  off  the  shelf  (COTS)  software.  In  addition,  the  measures  development  process 
would  be  presented  through  a  user-friendly  medium  that  would  require  very  little  if  any 
programmer  expertise  to  implement.  To  accomplish  this,  the  measures  development  process  was 
twofold:  development  of  the  measures  themselves  and  development  of  a  prototype  observer 
workstation  through  which  observers  could  develop  the  desired  measures  for  AARs.  Both  of 
these  efforts  are  described  in  more  detail  below. 

Automated  measures.  Once  the  definitions  and  output  formats  had  been  specified  for  the 
measures,  the  next  step  was  to  select  a  software  package  that  could  transform  the  data  that  would 
be  produced  by  the  DCA  into  the  desired  format.  For  all  the  DC"*!  efforts,  the  DCA  output 
provided  to  the  Project  Team  has  been  in  the  American  Standard  Code  for  Information 
Interchange  (ASCII)  format.  The  data  in  this  format  is  readily  imported  into  Microsoft®  Excel, 
which  has  been  used  to  create  tables  and  graphs.  The  Project  Team  decided  to  continue  to  use 
Excel  as  the  measures  creation  tool  for  several  reasons.  Excel  has  an  integrated  chart  wizard  that 
could  produce  the  majority  of  the  desired  output  formats  without  a  need  for  extensive 
programming.  The  chart  wizard  would  work  in  conjunction  with  a  macro  recorder  so  that  the 
same  type  of  chart  or  graph  could  be  repeatedly  generated  using  different  data  sets  without 
having  to  go  through  a  step-by-step  process  for  creating  each  output.  Changes  to  the  design  of 
the  measure  output  format  would  be  easy  to  make.  Charts  and  graphs  created  with  the  Excel 
wizard  could  be  further  customized  using  the  Microsoft®  Visual  Basic®  for  Applications 
programming  language,  which  is  included  with  Excel.  The  output  formats  could  then  be 
seamlessly  moved  to  other  Microsoft®  Office  products,  such  as  Word  for  reports  or  PowerPoint® 
presentation  graphics  program  for  presentations. 

Once  the  Project  Team  had  settled  on  using  Microsoft®  Office  as  the  baseline  software 
package,  development  of  the  measures  was  initiated  using  BCR  IV  data  available  from  the  DC"*!- 
2  project.  Early  on,  it  became  apparent  that  additional  data  would  be  required  to  support  the  new 
measures  that  had  been  designed  for  the  current  project.  The  Project  Team  generated  a  listing  of 
the  data  elements  that  would  be  required  to  support  each  measure.  Coordination  then  was 
effected  with  the  MMBL  to  obtain  the  required  additional  data.  Based  on  the  Project  Team’s 
experience  with  BCR  IV  data,  some  initial  decisions  were  made  to  facilitate  data  handling  and 
subsequent  analysis:  vehicle  locations  were  to  be  provided  at  5-minute  intervals;  status  of 
ammunition  and  fuel  were  to  be  provided  at  20-minute  intervals;  only  weapon  system  kills  were 
to  be  reported;  engagements  that  resulted  in  misses  or  less  than  a  catastrophic  kill  would  be 
excluded;  only  OPFOR  weapon  systems  that  were  detected  by  the  unit  and  its  subordinate 
elements  would  be  reported;  and  the  MMBL  would  package  the  data  into  three  separate  files  to 
prevent  the  mixing  of  incompatible  data  types. 

As  the  Project  Team  began  to  work  with  the  additional  data,  it  became  apparent  that  the  size 
of  the  data  files  would  exceed  the  capabilities  of  Microsoft®  Excel.  For  example,  in  the  sample 
BCR  IV  data  set,  an  average  mission  contained  175,000  rows  of  data  with  approximately  50 
columns  (i.e.,  variables).  Excel  could  only  handle  approximately  66,000  rows  of  data. 
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Therefore,  the  Project  Team  decided  to  use  Microsoft®  Access  for  data  manipulation  while 
continuing  to  use  Excel  to  create  the  charts  and  graphs. 

Once  the  MMBL  ASCII  data  files  were  imported  into  Microsoft®  Access,  the  Project  Team 
built  queries  for  each  of  the  picture,  graph,  and  table  output  formats.  Queries  in  Access  are 
designed  to  return  only  the  data  the  user  requires  from  the  larger  data  file.  Multiple  queries  were 
often  necessary  to  return  all  the  data  required  to  build  a  specific  measure.  The  Project  Team 
found  that  it  was  easier  to  manipulate  several  small  data  files  to  create  a  Microsoft®  Excel  chart 
or  graph  than  to  work  with  one  large  complex  data  file.  Additionally,  since  some  of  the 
measures  had  more  than  one  output  format,  even  more  query  files  would  be  required.  To  handle 
the  growing  number  of  queries,  the  Project  Team  created  Access  macros  that  would  control 
running  the  queries  associated  with  each  measure.  Therefore,  while  the  Project  Team  developed 
66  queries,  there  were  only  20  macros  -  one  for  each  of  the  20  developed  measures. 

Once  the  macros  were  developed,  a  form  was  constructed  in  Access  that  appears 
automatically  when  the  file  is  opened  (see  Figure  2).  The  form  does  three  things.  First,  it 
provides  the  user  a  link  to  Access  through  a  user-friendly  interface  that  will  be  incorporated  into 
the  Observer  Workstation.  In  this  way,  the  user  will  only  see  a  button  for  each  of  the  20 
measures,  and  not  all  the  queries  that  need  to  be  run  in  order  to  develop  a  measure.  Second,  it 
allows  the  user  to  set  the  time  for  which  the  output  data  files  would  be  created.  Once  the  user 
enters  the  time  and  clicks  on  a  button,  the  required  data  to  build  the  measure(s)  are  automatically 
output  to  Excel.  This  is  especially  critical  for  those  output  formats  that  are  essentially  snapshots 
in  time.  For  each  change  in  time,  queries  would  be  required  and  the  form  provides  a  simple 
mechanism  for  setting  the  time  for  all  queries.  Finally,  the  form  gives  the  user  an  option  of 
selecting  one  or  all  of  the  measures  at  one  time 

To  create  the  measure  output  formats,  the  Project  Team  began  by  using  the  Microsoft® 

Excel  chart  wizard,  which  was  described  earlier.  All  operator  actions  used  to  create  the  chart  or 
graph  were  captured  by  a  macro  recorder.  The  macros  were  then  modified  using  Microsoft® 
Visual  Basic®  in  order  to  achieve  the  desired  format.  In  many  instances,  this  meant  overriding 
the  various  Excel  default  chart  or  graph  characteristics.  For  example,  a  gridline  pattern  that 
represented  the  terrain  database  was  added  to  the  pictorial.  Standard  chart  colors  and  symbols 
were  aligned  with  Army  standards.  Background  and  foreground  colors  were  eliminated  or 
modified.  Additional  Visual  Basic®  software  code  had  to  be  written  to  work  around  some  of  the 
data  series  limitations  of  Excel.  An  example  of  an  Excel  macro  modified  with  Visual  Basic  is 
provided  in  Appendix  D.  The  Project  Team  also  developed  additional  Excel  files  that  were 
needed  to  support  the  output  format  creation. 
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Mote,  BLUFOR  blue  forceSj  CSS  combut  service  support^  OPFCR.  opposing 
forces;  PPT  =  Microsoft®  PowerPoint®. 

Figure  2.  Measures  development  screen  in  Mierosoft®  Access. 


Many  of  the  output  formats  incorporate  information  that  is  either  not  readily  available  from 
or  not  tracked  by  the  DCA.  For  example,  a  measure’s  pictorial  output  format  might  require  a 
depiction  of  the  unit’s  zone  of  action.  The  coordinates  for  the  zone  cannot  be  solely  determined 
by  obtaining  the  vertices  of  various  graphical  control  measures  plotted  on  the  plan  view  display 
(PVD),  due  to  a  multiplicity  of  linear  control  measures,  which  may  or  may  not  be  applicable  to 
the  current  mission.  On  the  other  hand,  an  observer  can  readily  determine  which  are  the 
appropriate  points  to  plot.  Various  reference  data  were  also  needed,  such  as  the  number  of 
expected  OPFOR  and  blue  force  (BLUFOR)  weapon  systems  in  the  mission,  maximum  effective 
ranges  of  weapons,  call  signs  for  various  staff  members,  weapon  system  categorization,  and  so 
forth. 


To  simplify  creating  the  output  formats,  a  similar  form  to  the  one  constructed  in  Microsoft® 
Access  was  developed  in  Microsoft®  Excel  so  users  could  select  the  measure  they  wanted  to 
create  by  clicking  a  button.  When  users  click  on  the  measure  they  want  to  create,  another  form 
appears  with  the  various  output  formats  available  for  that  particular  measure.  A  user  could  then 
choose  a  specific  format  or  select  all  output  formats.  If  only  one  output  format  is  available  for  a 
measure,  then  the  output  is  created  immediately. 


After  the  output  format  was  developed,  it  was  informally  evaluated  by  SMEs  who  were  not 
involved  in  the  measures  development  process  to  better  ensure  that  users  would  understand  the 
outputs  without  further  detailed  explanation.  Based  on  SME  input,  the  Project  Team  refined  the 
output  formats  by  adding  legends  and  changing  colors  until  they  were  immediately  interpretable. 


If  detailed  explanations  are  required  for  users  to  understand  the  purpose  of  the  picture,  it  will  be 
of  no  value  to  exercise  participants  in  an  AAR  (Meliza,  1996). 

The  next  step  was  to  provide  a  user-friendly  medium  through  which  this  complex  process 
could  take  place  and  where  the  final  product  could  be  either  printed  out  or  entered  into  a  slide 
presentation  to  be  used  during  an  AAR,  similar  to  an  “AAR  Presentation  Manager”  (Meliza, 
Bessemer,  &  Tan,  1994,  p.  45).  The  Project  Team  decided  that  the  best  way  to  accomplish  this 
goal  was  to  develop  a  prototype  Observer  Workstation.  The  development  process  for  the 
prototype  is  described  next. 

Observer  Workstation.  Figure  3  displays  the  entire  measures  development  process  and 
where  an  Observer  Workstation  would  fit  into  the  progression.  The  process  starts  with  the 
information  gathered  in  the  MMBL  by  the  data  logger  and  processed  through  the  DCA.  These 
refined  data  combined  with  information  inserted  by  observers  provide  the  information  needed  by 
the  Observer  Workstation  to  create  the  measures  automatically.  Once  the  measures  were 
developed  through  the  Observer  Workstation,  they  could  be  used  during  AARs  as  well  as  for  any 
reports  that  required  analysis  of  staff  performance. 


Note.  AAR  =  after  action  review;  BLEFR  =  Battle  Lab  Experiment  Final  Report; 

FCC2  =  Future  Combat  Command  and  Control;  PDU  =  protocol  data  unit;  SC"^  =  surrogate 
command,  control,  communications,  and  computers. 

Figure  3.  Automated  measures  development  process. 

Using  the  Microsoft®  PowerPoint®  presentation  graphics  program,  a  prototype  graphical 
user  interface  (GUI)  was  built  as  a  means  for  user-friendly  measure  output  format  creation.  This 
GUI  is  what  the  user  saw  when  interacting  with  the  Observer  Workstation.  Sample  screens  can 
be  seen  in  Figures  4  through  7. 
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Main  Menu 

Click  on  one  of  the  skill  dimensions  below  to  see  its  definition 
and  available  measures: 

Adaptability 

Communication 

Coordination 

Decision-Making 

Performance  Monitoring  &  Feedback 
Shared  Situational  Awareness 

Figure  4.  Main  menu  of  team  process  skill  dimensions  as  seen  in  the  graphical  user  interface. 


Gomplete  List  of  Automated 
Measures 


Air  Defense  Coverage 

Artillery  and  Counterfire  Radar  Coverage 

Common  Picture 

Counterreconnaissance  Effectiveness 
CSS  Available  Over  Time 
CSS  Locations 

Damage  to  BLUFOR/OPFOR  Systems 

Degradation  of  Forces 

Effects  of  Targeting 

Fire  Support  Coordination 


Fratricide/Coilateral  Damage 
Kill  Distance 
Maneuver  Battle  Sets 
Map  Area 

Multiple  Fire  Engagements 
Node  Locations 
Overlay  Use 
Sensor  Coverage 
Sensor-Shooter  Time  Lag 
Surprise  Attack 

Return  to  Menu  of  Skill  Dimensions 


Figure  5.  Complete  list  of  available  automated  measures  as  seen  in  the  graphical  user  interface. 


The  goal  for  the  Observer  Workstation  was  that  it  would  provide  a  user-friendly  way  for 
observers  with  no  programming  skills  to  produce  meaningful  output  for  feedback  to  the  staff 
during  AARs.  When  a  user  starts  the  program,  an  introductory  screen  appears.  After  entering 
the  program,  the  user  sees  the  instructions  screen.  The  user  can  choose  to  read  the  instructions 
and  then  move  on  to  the  main  menu,  which  contains  a  list  of  the  skill  dimensions  (see  Figure  4), 
or  if  familiar  with  the  program  and  the  available  measures,  the  user  can  select  to  view  the 
complete  list  of  automated  measures  (see  Figure  5). 
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Shared  Situational  Awareness 

Definition: 

The  process  by  which  team  members  develop  compatible 
models  of  the  team’s  internal  and  external  environment; 

Includes  skill  in  arriving  at  a  common  understanding  of  the 
situation  and  applying  appropriate  task  strategies 

Available  Measures: 

Common  Picture 
Map  Area 
Surprise  Attack 
Fire  Support  Coordination 
Air  Defense  Coverage 
Fratricide/Collateral  Damage 

Return  to  Menu  of  Skill  Dimensions 

Figure  6.  Definition  and  available  measures  for  the  shared  situational  awareness  team  process 
skill  dimension  as  seen  in  the  graphical  user  interface. 


Map  Area 

Definition: 

Square  kilometers  of  battlefield  displayed  for  each  staff  member  at 
specified  periods  of  time 

Available  Output 


Return  to  Menu  of  Shared  Situational  Return  to  List  of  Measures  Return  to  Menu  of  Skill  Dimensions 

Awareness  Measures 

Figure  7.  Definition  and  available  output  formats  for  the  map  area  measure  of  shared  situational 
awareness  as  seen  in  the  graphical  user  interface. 

If  the  user  chooses  the  first  option,  the  next  screen  will  provide  a  list  of  the  skill  dimensions 
available  for  measurement.  When  the  user  selects  one  of  the  skill  dimensions,  the  program  goes 
to  a  screen  that  provides  a  definition  of  the  skill  dimension  and  all  available  measures  for  that 
dimension  (see  Figure  6).  In  the  example  shown  in  Figure  6,  a  definition  is  provided  for  the  skill 
dimension.  Shared  Situational  Awareness,  along  with  the  six  available  measures.  When  the  user 
selects  one  of  the  measures,  the  program  provides  a  definition  of  the  measure  selected  as  well  as 
all  available  output  formats  for  that  particular  measure.  For  example,  as  seen  in  Figure  7,  the 
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definition  for  Map  Area  is  provided  along  with  a  sample  of  what  the  measure  output  will  look 
like  once  it  is  developed.  The  Map  Area  measure  can  only  be  produced  in  picture  format; 
however,  some  of  the  other  measures  have  more  than  one  format  available  and  the  user  can  click 
on  the  preferred  format. 

If  the  user  chooses  the  complete  list  of  available  automated  measures  (see  Figure  5),  the 
program  displays  all  20  of  the  available  measures  in  alphabetical  order.  When  the  user  selects 
one  of  the  measures,  the  program  will  go  to  that  particular  measure,  as  shown  in  Figure  7.  The 
list  of  all  available  measures  is  a  shortcut  for  those  users  who  are  already  familiar  with  the 
measures  and  know  which  ones  they  want  to  develop. 

Once  the  user  selects  a  measure  by  clicking  on  it.  Access  automatically  opens  and  the  form 
seen  in  Figure  2  comes  up.  The  user  again  selects  the  desired  measure  and  the  output  for  that 
measure  is  exported  to  Excel.  From  there,  the  user  again  selects  the  desired  measure  and  its 
output  format.  The  measure  output  is  then  created  and  saved  as  an  Excel  file. 

The  Observer  Workstation  is  designed  to  be  flexible  and  user-friendly.  It  allows  the  user  to 
go  almost  an)wvhere  within  the  program  at  any  given  time.  For  example,  if  the  user  selects  the 
wrong  skill  dimension  from  the  main  menu,  he  or  she  can  go  back  to  the  main  menu  and  choose 
another  skill  dimension  or  go  to  the  list  of  available  measures.  From  the  list  of  available 
measures  screen,  the  user  can  choose  to  go  to  the  main  menu  of  skill  dimensions.  If  the  user 
selects  a  particular  measure  and  decides  not  to  develop  it,  he  or  she  can  go  back  to  other 
measures  within  that  skill  dimension,  the  main  menu  of  skill  dimensions,  or  the  list  of  all 
available  measures. 


Measures  Implementation 

The  20  developed  automated  measures  along  with  the  Observer  Work  Station  (OWS)  were 
implemented  during  the  MMBL’s  FCC^  experiment.  The  workstation  was  physically  located  in 
the  data  analysis  area  where  the  Project  Team  could  readily  coordinate  with  the  MMBL 
programmers  on  FCC^  DCA  data  file  issues.  It  was  also  linked  to  the  MMBL’s  intranet  so  that 
data  files  could  be  electronically  downloaded.  The  Project  Team  had  access  to  an  SC'*  system 
where  zones  of  action,  OPFOR  strengths,  and  other  pertinent  data  could  be  extracted. 

As  the  measures  were  tested  during  initial  exercises,  a  number  of  changes  had  to  be  made. 
The  biggest  change  required  modifying  the  measures’  software  code  to  complete  processing 
when  a  DCA  source  file  unexpectedly  did  not  contain  any  data.  This  condition,  which  did  not 
occur  in  pre-implementation  development  testing,  was  a  result  of  changes  in  unit  organization 
and  equipment  that  the  Project  Team  had  not  anticipated  prior  to  the  start  of  the  experiment. 
Processing  of  raw  DCA  data  files  for  input  to  the  OWS  also  took  longer  than  expeeted.  In  the 
case  of  a  full  mission’s  data  set,  it  took  approximately  2  hours  to  produce  the  ASCII  files  needed 
for  the  measures.  It  then  took  another  30  to  45  minutes  to  import  the  data  into  Access  hosted  on 
the  OWS.  The  net  effect  was  that  sample  outputs  from  the  measures  were  not  available  in  time 
for  use  in  AARs  during  the  experiment.  Some  sample  results  were  shared  with  MMBL 
personnel  and  contractors  during  the  experiment  to  gain  some  informal  feedback  on  formatting. 
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A  full  set  of  measure  outputs  was  presented  to  the  SAG  for  review  and  to  the  MMBL  for  Battle 
Lab  Experiment  Final  Report  (BLEFR)  input  after  the  FCC“  experiment  was  concluded. 

Results  and  Discussion 

To  provide  trainers  with  a  variety  of  means  to  depict  staff  performance,  33  output  formats 
were  designed  to  support  the  20  automated  measures.  This  section  provides  samples  of  each 
output  format.  The  discussion  of  each  output  format  is  organized  into  three  parts: 

•  A  description  is  provided  of  what  information  is  being  presented  along  with  any  filters 
that  were  applied  to  the  source  database  to  obtain  the  data  elements  required  to  create  the 
output.  A  rationale  for  the  presenting  the  information  is  also  provided. 


•  The  measure’s  result  is  discussed  briefly.  The  results  should  not  be  viewed  as  a 
reflection  of  the  unit’s  performance.  As  described  earlier  in  this  report,  the  intent  of  the 
FCC^  experimentation  was  to  gain  insights  into  future  issues,  not  evaluate  specific 
performance  of  a  unit  using  conceptual  weapons,  organization,  or  doctrine.  Performance 
standards  for  this  type  of  unit  have  not  been  established. 

•  Comments  on  the  utility  of  the  output  format  and  recommendations  on  how  it  could  be 
improved  are  also  provided. 


Readers  are  further  advised  that  the  original  output  formats  provided  were  developed  with 
color  and  shape  as  primary  identifying  characteristics.  Generally,  blue  squares  are  used  to 
denote  BLUFOR  while  red  diamonds  are  used  to  denote  OPFOR.  Although  the  shapes  are  still 
identifiable,  colors  cannot  be  reproduced  in  this  report.  In  the  electronic  versions  of  the  output 
formats,  the  user  can  modify  the  characteristics  or  delete  any  data  series  to  clarify  the 
presentation  of  information.  Again,  this  capability  cannot  be  replicated  in  this  report.  Full  color 
samples  or  electronic  files  of  these  measures  and  output  formats  can  be  obtained  from  the 
MMBL. 

For  the  picture  output  formats,  the  Project  Team  created  a  grid  system  to  represent  the 
terrain  database  used  during  the  trials.  The  terrain  database  covers  an  area  1 84  kilometers  by 
130  kilometers.  The  grid  represents  10,000-meter  intervals  and  covers  the  entire  terrain.  The 
Project  Team  has  not  yet  integrated  tactical  ground  maps  into  pictorial  representations  due  to 
scaling  problems  and  the  potential  for  map  information  to  clutter  the  presentation  of  critical 
information.  The  unit  zone  of  action  for  each  trial  is  also  drawn  automatically  from  data  in  a 
table  created  by  an  observer.  The  zone  of  action  represents  the  geographical  area  where  the 
majority  of  the  unit’s  maneuver  forces  were  positioned  and  where  close  combat  with  the  OPFOR 
was  expected.  The  pictorial  representation  of  the  unit  zone  of  action  along  with  an  arrow 
indicating  the  OPFOR  direction  of  movement  has  been  found  to  adequately  orient  a  training 
participant  to  the  activity  being  depicted. 

The  sample  results  provided  here  are  from  Trial  1  during  FCC^.  The  results  of  all  four  FCC^ 
trials  were  provided  to  the  MMBL  for  use  in  the  FCC^  BLEFR.  Results  are  categorized  by  the 
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team  process  skill  dimensions  identified  earlier.  The  mission  of  the  unit,  a  reinforced  battalion 
equivalent,  was  to  defend  against  an  attack  by  several  OPFOR  mechanized  infantry  brigades, 
reinforced  with  Corps-level  artillery.  The  area  to  be  defended  is  approximated  by  the  unit  zone 
of  action,  which  is  depicted  on  most  of  the  pictures.  Trial  1,  which  started  at  0800  local  time, 
lasted  approximately  7  1/2  hours. 


Adaptability 


CSS  Locations 

The  picture  for  this  measure  (Figure  8)  was  designed  to  show  key  vehicle  locations  relative 
to  ammunition  and  fuel  resupply  vehicles.  The  platoon  leader  vehicle  locations  were  used  to 
represent  the  location  of  all  major  combat  systems  in  the  battalion.  The  Project  Team  decided 
that  showing  all  of  the  vehicles  would  clutter  the  presentation.  Normally,  all  of  the  vehicles 
within  a  maneuver  platoon  are  located  within  visual  range  of  each  other,  so  the  location  of  the 
platoon  leader  vehicle  was  thought  to  be  an  accurate  representation  of  where  the  other  vehicles 
were  located.  Also  shown  are  the  ammunition  and  fuel  resupply  vehicles.  The  Project  Team 
used  symbols  for  the  resupply  vehicles  that  were  slightly  larger  than  the  platoon  leader  vehicle 
symbols  to  accentuate  their  location  and  to  keep  them  from  being  covered  by  another  vehicle 
type  when  they  were  plotted.  Data  for  the  vehicle  locations  were  available  at  5-minute  intervals 
and  come  from  the  same  source  that  would  be  used  to  show  vehicle  unit  marking,  type,  and 
location  on  the  unit  member’s  PVD.  There  were  a  total  of  10  ammunition  carriers  and  6  fuel 
carriers  that  could  be  depicted.  Figure  8  depicts  the  locations  of  one  ammunition  and  two  fuel 
carriers. 

The  rationale  for  this  measure  was  to  identify  if  the  staffs  mission  planning  had 
incorporated  sufficient  flexibility  to  respond  to  unanticipated  requirements,  such  as  a  platoon 
running  out  of  fuel  or  ammunition  during  heavy  combat  or  at  a  critical  point  in  the  mission.  If 
the  resupply  vehicles  were  located  where  they  were  needed  when  they  were  needed,  even  from 
an  unforeseen  requirement,  an  inference  could  be  made  that  the  staff  had  adequately  planned 
resupply. 

At  the  time  of  this  picture  (approximately  1  hour  after  the  trial  started),  there  had  been 
insufficient  activity  to  determine  if  the  ammunition  and  fuel  resupply  vehicles  were  adequately 
positioned  to  support  the  maneuver  platoons.  The  fuel  vehicles  were  in  close  proximity  to  the 
platoons,  but  the  one  ammunition  vehicle  depicted  was  at  least  1 0,000  meters  or  about  1 5 
minutes  travel  time  from  the  nearest  platoon. 

The  picture  does  provide  a  quick  view  of  where  the  resupply  vehicles  are  in  relationship  to 
the  maneuver  platoons  within  the  unit.  It  does  not  provide  the  ammunition  or  fuel  status  for  the 
subordinate  units,  which  would  indicate  if  the  resupply  vehicles  are  positioned  appropriately. 
Adding  subordinate  unit  supply  indicators  to  the  picture  may  improve  its  usefulness. 
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Figure  8.  Combat  service  support  locations  at  0900. 

Node  Locations 

The  picture  designed  for  this  measure  (see  Figure  9)  shoves  the  location  of  the  unit’s 
commander  and  staff  vehicles  (nodes)  in  relationship  to  the  location  of  vehicles  of  the  maneuver 
platoon  leaders,  scout  platoon  leader,  and  platoon  sergeant.  The  experimental  staff  in  FCC^, 
including  the  commander,  comprised  14  officers  and  non-commissioned  officers  located  in  four 
vehicles.  The  commander  had  two  assistant  staff  officers  in  his  vehicle  as  did  the  deputy 
commander  in  his.  The  third  staff  vehicle  (Control  1)  was  primarily  concerned  with  controlling 
and  monitoring  the  unit’s  indirect  fires  and  engineer  support.  The  fourth  staff  vehicle  (Control  2) 
was  responsible  for  monitoring  the  unit’s  logistical  situation  and  controlling  maintenance  and 
resupply  activities  during  FCC^.  The  Project  Team  used  symbols  that  were  slightly  larger  than 
the  platoon  leader  vehicles  to  denote  the  Control  1  and  Control  2  vehicles.  The  Commander 
(Cdr)  and  Deputy  Commander  (DepCdr)  vehicle  designators  were  increased  in  size  over  the 
control  vehicle  symbols  so  that  if  the  command  vehicle  were  collocated  with  the  control  vehicles 
they  could  be  differentiated.  Data  for  the  vehicle  locations  were  available  in  5-minute  intervals 
and  come  from  the  same  source  that  would  be  used  to  show  vehicle  unit  marking,  type,  and 
location  on  the  unit  member’s  PVD. 

The  rationale  for  this  measure  was  that  the  location  of  the  unit’s  command  and  control  nodes 
may  indicate  the  ability  of  the  staff  to  handle  different  requirements  simultaneously  while 
keeping  positioned  to  maintain  eommunications  with  all  subordinate  elements,  and  maintaining 
operational  and  physical  security. 
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Note.  OPFOR  =  opposing  forces. 

Figure  9.  Node  locations  at  0900. 

The  locations  of  the  nodes  indicate  that  the  Cdr  and  DepCdr  nodes  were  centrally  located  in 
the  zone  of  action  and  in  close  proximity  to  one  another.  Control  2  was  likewise  centrally 
located.  Control  1  was  located  well  outside  of  the  zone  of  action.  With  the  unit’s  SC"*  system 
capabilities,  the  ability  to  maintain  communications  was  not  dependent  on  proximity. 

Operational  and  physical  security  was  a  consideration,  and  ideally  Control  1  should  have  been 
located  closer  to  the  unit’s  subordinate  elements. 

The  picture  provides  a  quick  reference  as  to  where  the  commander  and  staff  were  located  at 
any  point  during  the  mission.  If  communications  among  staff  vehicles  and  between  the  staff  and 
subordinate  units  is  affected  by  terrain  or  distance,  then  a  communications  pattern  indicator 
similar  to  a  range  template  might  be  useful.  If  physical  security  is  a  primary  consideration, 
adding  detected  OPFOR  locations  might  also  be  warranted. 

Communication 


Overlay  Use 

Table  4  shows  which  staff  officers  and  team  commanders  had  the  same  operational  overlays 
available  on  their  SC'*  system  as  did  the  unit  commander.  The  source  of  the  data  is  a  record  of 
each  time  the  PVD  Overlay  Editor  is  used  to  create  an  overlay,  or  to  display  or  turn  off  an 
existing  overlay.  The  recorder  also  captures  the  time,  the  name  of  overlay  file  being 
manipulated,  and  the  radio  call  sign  of  the  soldier  working  with  the  overlay.  For  Trial  1,  there 
were  approximately  7,700  instances  where  the  PVD  Editor  was  used.  Using  the  unit 
commander’s  radio  call  sign,  the  list  was  filtered  several  times,  to  identify  the  overlay  files  the 
commander  used.  That  list  was  compared  with  the  overlay  files  that  the  staff  officers  in  his 
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vehicle  used,  with  other  staff  members’  overlay  files,  and  with  his  major  subordinate  team 
commander’s  overlay  files. 

The  unit  operations  orders  (OPORDs)  were  transmitted  to  the  staff  and  subordinate  units 
through  the  use  of  Whiteboard  files  and  through  the  use  of  PVD  overlay  files.  Use  of  the  PVD 
overlay  file  that  contains  the  OPORD  operations  overlay  has  been  found  by  the  Project  Team, 
based  on  previous  experimentation,  to  be  a  reliable  indicator  of  who  has  received  the  OPORD. 
The  comparison  between  the  commander  and  his  staff,  and  between  the  commander  and  his 
subordinate  commanders  may  indicate  whether  there  is  a  potential  for  miscommunication  and  a 
breakdown  of  situational  awareness  within  the  unit  if  the  commanders  and  staff  are  not  using  the 
same  overlays. 

Table  4 


Overlay  Use 


Cougar6  Overlays  -  Name  - 

Command  1 

Command  2 

Couear62 

Couear69 

Couear3 

Couear32 

Cousar35 

Defend  Overlay  1 

y 

y 

Defend  Overlay  2 

y 

y 

Defend  Overlay  3 

Attack  Overlay  1 

y 

y 

y 

y 

Attack  Overlay  2 

y 

Attack  Overlay  3 

Passage  of  Lines  Overlay  1 

y 

y 

y 

y 

Passage  of  Lines  Overlay  2 
Unit  Boundary  Overlay 

y 

Control  1 

Control  2 

Cougar6  Overlays  -  Name  - 

Outlaw6 

Outlaw2 

Outlaw3 

Head  Hunter6 

Head  Hunter2 

Head  Hunter3 

Defend  Overlay  1 

y 

y 

y 

y 

Defend  Overlay  2 

y 

y 

Defend  Overlay  3 

y 

y 

y 

y 

y 

Attack  Overlay  1 

Attack  Overlay  2 

y 

y 

Attack  Overlay  3 

Passage  of  Lines  Overlay  1 

y 

Passage  of  Lines  Overlay  2 
Unit  Boundary  Overlay 

y 

(table  continues) 
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Table  4  (Continued) 


Cougar6  Overlays  -  Name 

Defend  Overlay  1 
Defend  Overlay  2 
Defend  Overlay  3 
Attack  Overlay  1 
Attack  Overlay  2 
Attack  Overlay  3 
Passage  of  Lines  Overlay  1 
Passage  of  Lines  Overlay  2 
Unit  Boundary  Overlay 


Team  Commanders 


Easle6 

Fox6 

Ghost6 

Hawk6 

y/^ 

Among  the  staff,  the  senior  officer  in  each  node  (Headhunterb  and  Outlaw6)  were  also  using 
the  same  overlay  files  as  Cougarb  and  CougerS,  the  Deputy  Commander.  Interestingly,  only  one 
of  the  commander’s  two  assistants,  Cougar69,  had  the  same  overlay  files  as  he  did.  In  the 
Command  2  node,  only  the  Deputy  Commander  had  overlay  files  related  by  title  to  the  Defend 
mission  of  the  unit.  The  team  commanders  (identified  by  the  suffix  “6”  in  the  radio  call  sign), 
with  the  exception  of  Ghostb,  had  most  of  the  same  overlay  files  as  the  unit  commander, 

Cougarb. 

This  measure  identifies  which  members  of  the  staff  have  the  same  overlay  files  as  the  unit 
commander.  The  Project  Team  had  to  construct  these  tables  manually,  since  a  reliable  way  to 
automatically  match  file  names  among  the  staff  had  not  been  developed  by  the  time  this  report 
was  prepared.  By  relying  exclusively  on  file  names  as  the  basis  for  the  comparison,  there  is  a 
potential  problem  with  identifying  the  right  overlay.  For  example,  if  two  staff  members  were 
working  on  the  same  overlay  file  simultaneously  and  closed  the  files  to  a  common  file  server 
without  changing  or  modifying  the  file  name,  the  last-saved  file  would  overwrite  the  first-saved 
file,  even  though  the  contents  of  the  files  might  be  different.  The  first  staff  officer  would  assume 
that  his  work  on  the  overlay  file  had  been  saved  when  in  fact  it  was  not.  Other  staff  officers 
might  not  even  be  aware  that  changes  had  been  made  to  the  overlay  file  and  that  they  might  not 
have  the  most  current  file.  To  counteract  this  possibility,  the  unit  staff  would  have  to  establish  a 
standing  operating  procedure  (SOP)  for  naming  OPORD  overlays  and  tracking  modifications  to 
them. 


Coordination 


Damage  to  BLUFOR/OPFOR  Systems 

There  are  two  bar  graphs  and  two  cumulative  line  graphs  associated  with  this  measure.  The 
Project  Team  attempted  to  integrate  BLUFOR  and  OPFOR  data  into  single  bar  and  line  graphs, 
but  the  presentation  of  information  was  overly  complex.  The  data  used  to  create  the  output 
formats  are  the  type  of  weapon  system  that  was  killed,  the  type  of  weapon  system  that  killed  it, 
and  the  time  the  weapon  system  was  killed.  The  killed  weapon  system  is  classified  into  one  of 
four  groups  through  the  use  of  a  look-up  table  that  is  constructed  ahead  of  the  exercise  by  an 
observer  on  the  OWS.  The  four  groups  are:  “Air  Defense”  -  weapon  systems  with  a  primary 
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role  of  killing  helicopters  and  other  aircraft;  “Artillery”  -  indirect  fire  weapon  systems  whose 
primary  mission  is  to  attack  areas  of  OPFOR  concentration  throughout  the  depths  of  the 
battlefield;  “IFV_APC”  -  ground  combat  vehicles  whose  primary  mission  is  carry  infantry 
soldiers  and  which  may  or  may  not  have  organic  weapon  systems  capable  of  defeating  tanks;  and 
“Tank_FV”  -  tanks  and  other  ground  combat  vehicles  that  have  organic  weapon  systems  capable 
of  defeating  tanks  with  direct  fire,  but  whose  primary  mission  is  not  to  carry  infantry  soldiers. 

For  those  weapon  systems,  which  did  not  fall  into  these  four  groupings,  like  attack  helicopters  or 
dismounted  infantry,  the  designation  “#N/A”  is  automatically  assigned  by  Microsoft®  Excel. 
Each  weapon  system  is  also  classified  as  providing  direct  fire  (line  of  sight)  which  means  that  the 
firing  weapon  system  can  see  its  target,  or  indirect  fire  (beyond  line  of  sight  or  non-line  of  sight). 

The  rationale  for  this  measure  was  two-fold.  First,  the  staff  is  traditionally  responsible  for 
planning  indirect  fires  in  support  of  the  commander’s  concept  of  the  operation.  More  indirect 
fire  kills  than  direct  fire  kills  may  indicate  that  the  staff  has  effectively  coordinated  fires.  Also, 
the  distribution  of  losses  in  the  unit  due  to  indirect  fire  and  indirect  fire  from  the  OPFOR  may 
indicate  whether  the  staff  was  successful  coordinating  types  of  fire.  Indirect  weapon  systems, 
due  to  their  longer  ranges,  are  habitually  employed  to  attack  the  other  side’s  indirect  fire  weapon 
systems.  Second,  in  the  expected  progression  of  a  more  conventional  combat  operation,  the  first 
weapon  systems  destroyed  are  those  that  are  in  close  proximity  or  direct  fire  range.  As  the 
operation  unfolds,  artillery  and  air  defense  weapon  systems,  which  are  generally  located  farther 
away,  are  attacked  later.  If  all  classes  of  weapons  are  being  attacked  at  the  same  time,  the  staff 
may  be  effectively  coordinating  fires  throughout  the  battlefield. 

In  Figure  10,  the  data  indicate  that  the  majority  of  losses  suffered  by  the  OPFOR  were  due 
to  indirect  fire  weapons.  The  only  type  of  weapon  system  that  experienced  significant  losses  due 
to  direct  fire  were  Tank  FVs,  which  would  be  expected  to  be  in  the  lead  elements  of  an  OPFOR 
attack.  IFV_APCs  also  experienced  losses  due  to  direct  fire,  but  not  to  the  same  extent  that 
Tank_FVs  were  experiencing. 
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Figure  10.  Opposing  force  damage  from  blue  forces. 

The  BLUFOR  Damage  bar  graph  (Figure  1 1)  also  indicates  that  indirect  fire  was  the 
primary  cause  of  losses  to  the  unit.  More  indirect  fire  systems  were  lost  than  direct  fire  systems, 
which  may  indicate  that  those  systems  were  easier  to  target  by  the  OPFOR  because  they  moved 
less  often,  or  that  they  were  positioned  too  far  forward,  which  increased  their  vulnerability. 
Further  analysis  would  be  required  to  find  the  reason  for  this  result. 


29 


Figure  11.  Blue  force  damage  from  opposing  forces. 

The  OPFOR  Destruction  line  graph  (Figure  12)  shows  that  Tank_FVs,  IFV_APCs,  and 
artillery  systems  were  being  destroyed  early  in  the  mission  at  approximately  the  same  rate.  This 
may  indicate  that  the  staff  was  able  to  coordinate  fires  throughout  the  depth  of  the  battlefield. 
The  drop  in  the  rate  of  loss  on  Tank  FVs  at  the  2-hour  mark,  and  the  rate  of  loss  for  Artillery 
and  Air  Defense  systems,  starting  around  the  3-hour  mark,  may  indicate  that  almost  all  of  the 
OPFOR  targets  in  these  classes  of  weapons  had  been  destroyed  by  that  point  in  time,  these 
systems  were  out  of  range,  or  were  not  being  detected  by  the  unit. 
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Figure  12.  Opposing  forces  destruction  by  weapon  classification  over  time. 

The  BLUFOR  Destruction  line  graph  (Figure  13)  shows  that  the  early  rate  of  loss  in  the 
Artillery  category  carried  throughout  the  trial.  Additional  analysis  is  required  to  explain  this 
trend.  The  expected  pattern  would  be  that  those  elements  of  the  unit  that  were  closest  to  the 
OPFOR  would  sustain  the  most  losses  early  in  the  fight.  That  does  not  appear  to  be  the  case  in 
this  trial. 

Overall,  the  bar  graphs  appear  more  informative  than  the  line  graphs.  The  bar  graphs  show 
what  types  of  weapons  (direct  or  indirect)  caused  the  most  casualties.  The  line  graphs,  while 
showing  at  what  time  losses  were  occurring,  would  also  require  an  observer  to  interpret  why  the 
losses  were  happening  when  they  did.  As  they  are  currently  constructed,  the  line  graph  could  not 
stand  alone,  and  this  is  an  important  goal  for  all  of  the  measure  output  formats. 
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Figure  13.  Blue  force  destruction  by  weapon  classification  over  time. 

Degradation  of  Forces 

The  graph  developed  for  this  measure  (Figure  14)  provides  a  comparison  between  the 
combat  strength  of  both  sides  during  the  trial.  At  5-minute  intervals,  the  cumulative  number  of 
major  combat  systems  that  had  been  killed  to  that  point  is  recorded.  Those  data  are 
automatically  integrated  into  a  worksheet  that  has  the  number  of  systems  with  which  each  side 
started  the  mission.  Losses  are  subtracted  from  the  starting  total  and  a  percentage  of  the  force 
that  is  still  alive  is  then  calculated.  The  data  are  then  plotted  along  a  timeline  from  when  the 
mission  began. 

A  rationale  for  this  measure  is  that  rapid  destruction  of  the  OPFOR  reduces  the  risk  of  losses 
to  the  friendly  unit.  The  rate  of  loss  also  may  indicate  battle  tempo  during  the  mission  and 
whether  the  staff  is  coordinating  the  efforts  of  the  unit  to  inflict  the  maximum  number  of  losses 
in  the  shortest  period  of  time. 

Results  from  Trial  1  indicate  that  for  the  first  2  hours,  the  rates  of  loss  between  the  OPFOR 
and  friendly  forces  were  comparable  and  moderate.  After  2  hours,  the  rate  of  loss  for  the 
OPFOR  substantially  increased,  while  the  rate  of  loss  to  the  friendly  side  decreased  considerably. 
Upward  spikes  in  the  lines  for  both  sides  indicate  where  either  destroyed  systems  were  replaced 
or  additional  forces  were  committed. 

The  graph  provides  a  quick  comparison  of  the  strengths  of  the  two  sides  over  time.  While  it 
covers  critical  combat  systems,  it  does  not  break  the  losses  down  into  categories  of  combat 
systems  as  does  the  Damage  to  BLUFOR/OPFOR  Systems  measure  output  formats  described 
earlier.  A  potential  improvement  might  be  to  add  lines  to  indicate  when  the  units  move  from 
green  to  amber  to  red  to  black. 
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Figure  14.  Degradation  of  forces  over  time. 

Counterreconnaissance  Effectiveness 

The  picture  developed  for  this  measure  (Figure  15)  reflects  the  location  where  OPFOR 
ground  reconnaissance  vehicles  were  first  detected  and  the  location  where  they  were  killed.  In 
order  to  visually  link  these  two  points  a  straight  line  is  plotted  between  them;  the  line  does  not 
reflect  the  actual  route  the  reconnaissance  vehicle  was  following  before  it  was  killed.  The  legend 
for  this  picture  identifies  the  pairing  of  the  location  where  the  vehicle  was  detected  and  the 
location  where  it  was  killed  as  a  “Series.”  This  identification  is  assigned  automatically.  While 
the  legend  could  have  been  excluded,  the  number  of  the  series  could  be  used  to  identify  how 
vehicles  were  being  tracked,  especially  in  those  instances  where  lines  or  vehicle  locations  are 
being  superimposed  on  one  another.  In  this  case,  eight  vehicles  were  being  tracked  (the  number 
of  series  divided  by  two). 

This  measure  may  indicate  if  the  staff  was  coordinating  fires  effectively  to  negate  the 
OPFOR  ground  reconnaissance  effort.  If  the  OPFOR  cannot  detect  the  unit,  the  unit  reduces  its 
vulnerability.  The  OPFOR  ground  reconnaissance  vehicles  are  high  priority,  high  payoff  targets 
(FIPTs)  that,  doctrinally,  should  be  among  the  first  targets  attacked  after  they  have  been  detected. 

The  data  presented  suggest  that  OPFOR  ground  reconnaissance  vehicles,  while  detected 
soon  after  they  had  moved  into  the  unit’s  zone  of  action,  were  not  killed  until  they  had  an 
opportunity  to  move,  on  average,  about  10,000  meters  farther  into  the  zone.  What  is  not 
available  in  this  output  format  is  the  time  that  the  OPFOR  vehicles  were  detected  and 
subsequently  killed.  Those  data  would  indicate  how  much  time  the  OPFOR  would  have  had  to 
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detect  the  unit.  Also  missing  are  unit  positions,  which  again  would  provide  an  indication  of  how 
much  information  the  OPFOR  could  have  gained  before  they  were  destroyed. 


Figure  15.  Counterreconnaissance  effectiveness. 

Artillery  and  Counterfire  Radar  Coverage 

The  picture  designed  for  this  measure  (Figure  16)  depicts  the  range  templates  of  the  indirect 
fire  weapons  that  are  controlled  by  the  unit  staff.  There  are  a  total  of  12  systems  -  six  missiles, 
three  rockets,  and  three  1 55mm  howitzers  that  could  be  positioned  by  the  staff.  Location  for 
individual  systems  was  available  at  5 -minute  intervals.  To  create  the  range  templates,  data  for 
each  individual  weapon  system  are  automatically  posted  to  a  worksheet,  which  plots  the  points 
necessary  to  create  the  circle  representing  the  range  template.  If  a  particular  system  has  been 
destroyed,  the  location  of  the  system  is  calculated  as  being  at  0,0  and  the  range  template  for  it  is 
drawn  around  that  point,  as  indicated  in  the  lower  left  portion  of  Figure  16.  The  legend  provides 
the  unit  identification  number  for  the  individual  systems.  Units  being  supported  by  the  weapon 
systems  are  depicted  by  plotting  the  maneuver  platoon  leader  positions.  The  OPFOR  depicted 
represent  detected  OPFOR  systems.  The  OPFOR  location  data  were  available  at  20-minute 
intervals. 

This  measure  is  designed  to  provide  information  on  whether  the  staff  is  effectively 
coordinating  the  indirect  fire  assets  it  directly  controls  to  support  both  its  subordinate  units  and 
the  commander’s  intent  and  concept  of  the  operation.  Typically,  indirect  fire  weapon  systems, 
while  geographically  dispersed,  are  positioned  where  they  can  mass  timely  fires  at  decisive 
points  in  the  operation,  and  attack  targets  of  opportunity  while  reducing  BLUFOR  exposure  to 
OPFOR  counterartillery  fires. 
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Figure  16.  Artillery  coverage. 

The  results  depicted  in  Figure  16  show  that  the  missile  systems  are  positioned  to  engage 
targets  throughout  the  unit’s  zone  of  action  and  most  of  the  detected  OPFOR  targets  outside  of 
the  zone.  Several  of  the  unit’s  weapon  systems  are  shown  as  being  destroyed  or  not  available 
(lower  left  portion  of  Figure  16).  This  may  be  explained  by  the  loss  of  these  systems  to  OPFOR 
indirect  fire,  which  was  noted  under  the  results  obtained  by  the  Damage  to  BLUFOR/OPFOR 
Systems  measure. 

The  output  format  provides  a  clear  depiction  of  what  targets  could  be  engaged  by  the 
weapon  systems.  The  legend  needs  to  be  simplified  to  relate  the  color  of  the  range  template  to 
the  type  of  weapon  systems  being  represented  rather  than  the  unit  vehicle  marking  number. 
Another  improvement  would  be  to  automatically  remove  individual  weapon  systems  that  were 
destroyed  or  not  available  from  the  graph  which  would  prevent  their  range  templates  from  being 
plotted  around  the  0,0  point  and  remove  their  unit  vehicle  marking  number  from  the  legend. 

Kill  Distance 

There  are  two  output  formats  for  this  measure  -  a  line  graph  and  a  table.  The  line  graph 
(Figure  17)  is  generated  by  categorizing  the  range  from  the  shooter  to  the  target  for  each  OPFOR 
weapon  systems  killed.  To  facilitate  the  presentation  of  information,  12  range  categories  were 
created.  Initially,  the  line  graph  was  designed  to  report  information  for  those  weapon  systems 
that  were  directly  controlled  by  the  staff  (missile,  rocket,  and  1 55mm  howitzer).  Since  the 
number  of  kills  for  those  systems  was  limited  during  the  experiment,  three  additional  weapon 
systems  were  added  (indirect  fire  [IDF_GUN],  mortar  [MTR],  and  precision  attack  missile 
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[PAM]).  The  number  of  kills  by  the  various  weapon  systems  for  each  range  category  was  then 
computed  and  plotted.  A  linear  trendline  was  then  created  for  each  weapon  system.  As  seen  in 
Figure  17,  just  three  weapon  systems  accounted  for  all  of  the  Trial  1  kills,  and  just  two  systems 
had  a  sufficient  number  of  kills  to  generate  a  trendline.  All  of  the  mortar  systems  (vehicle  robot 
mortar  [VEH_ROB_MTR])  kills  occurred  in  the  same  range  category,  which  did  not  generate  a 
discernible  trend  line  in  the  graph. 

The  rationale  for  this  output  format  was  that  if  the  staff  was  effectively  coordinating  the 
fires  of  the  weapons  it  directly  controlled,  then  the  majority  of  the  kills  they  obtained  should  be 
nearer  the  maximum  effective  range  of  the  weapon  rather  than  the  minimum  effective  range  of 
the  weapon. 

The  results  obtained  by  this  measure  indicate  that  the  unit  was  not  maximizing  the 
effectiveness  of  the  IDF_GUN  weapon  systems;  no  clear  trend  is  discernible  for  the  PAM 
weapon  systems.  Both  of  these  weapon  systems  were  controlled  by  the  unit’s  subordinate  teams. 
If  all  of  the  weapon  systems  in  the  unit  had  been  plotted,  the  graph  might  have  been  too  crowded. 


Figure  1 8  is  the  other  output  format  available  for  this  measure.  The  average  engagement 
range  for  each  weapon  system  is  automatically  inserted  into  the  table.  To  provide  a  comparison 
with  OPFOR  performance,  the  data  for  their  weapon  systems  are  also  provided.  Data  for  all 
weapon  systems  involved  in  Trial  1  are  reflected  in  this  table.  While  this  table  provides  the 
maximum  effective  ranges  for  weapon  systems  which  are  not  provided  in  the  graph,  one  or  two 
engagements  at  very  long  ranges  could  skew  the  results  somewhat.  The  graph  provides  data  that 
indicate  how  the  engagements  were  distributed  across  the  effective  range  of  each  system. 
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The  data  in  the  table  confirm  the  data  interpretation  of  the  graph;  that  is,  the  IDF  GUN 
system  was  not  being  employed  at  ranges  commensurate  with  its  capabilities,  while  the  PAM 
system  average  kill  distance  was  mid-point  in  its  effectiveness  range. 


BLUFOR 

1  Weapon  Svstem  Maximum  Effective  Ranae 

Averaae  Ranae 

Indirect 

VEH_ROB_IDF^GUN 

50000 

15526 

VEH_ROB_PAM 

50000 

20827 

VEH_ROB_LAM 

50000 

#N/A 

VEH_ROBJDF_RKT 

50000 

#N/A 

VEH_ROB_155 

27300 

#N/A 

VEH_ROB_MTR 

50000 

6012 

Direct 

VEH_ROB_IDF_GUN{KE) 

50000 

#N/A 

VEH_MAN  D_TRP_TRANS 

2000 

1818 

VEH  MAND  TRP  TRANS 

CM  2000 

979 

VEH^MAND„C2 

1000 

#N/A 

OPFOR 

I  Weaoon  Svstem  Maximum  Effective  Ranae 

Averaae  Ranae 

indirect 

ELDO  VEH  2S19 

24700 

#N/A 

ELDO  VEH  2S23 

24700 

#N/A 

ELDO  VEH  2S9 

6000 

#N/A 

ELDO_VEH_BM_21 

20300 

#N/A 

ELDO_VEH_BM_22 

140000 

30253 

nirpr.t 

ELDO_VEH_BMP_3 

4000 

#N/A 

ELDO  VEH_BRDM_2 

2000 

#N/A 

ELDO_VEH_BRM_3 

4000 

2059 

ELDO  VEH_BTR_80 

4000 

#N/A 

ELDO  VEH  BTR  90 

1000 

252 

ELDO  VEH_T72_MP 

1000 

2104 

ELDO_VEH_T80_UD 

1000 

#N/A 

ELDO  VEH  T90 

1000 

2436 

Note.  BLUFOR  =  blue  forces;  OPFOR  =  opposing  forces. 


Figure  18.  Kill  distance  table. 


Improvements  to  the  output  formats  for  this  measure  might  include  providing  an  indicator  of 
the  maximum  effective  range  for  the  particular  weapon  system  being  plotted  on  the  graph. 
Including  the  number  of  kills  for  each  weapon  system  in  the  table  would  inform  the  reader  of  the 
sample  size  and  give  more  power  to  the  results,  especially  if  the  sample  size  is  large. 
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Sensor-Shooter  Time  Lag 

There  are  two  output  formats  associated  with  this  measure  -  a  graph  and  a  picture.  The 
graph  (Figure  19)  is  created  from  data  that  provide  the  time  the  target  was  spotted  and  the  time  it 
was  killed.  The  difference  between  the  two  times  is  then  graphed,  based  on  seven  30-minute 
time  categories.  The  30-minute  categories  are  based  on  the  Project  Team’s  previous  experience 
with  experiments  similar  to  FCC^,  where  the  preponderance  of  targets  was  killed  within  1 80 
minutes  of  being  detected.  This  measure  is  used  as  an  indicator  of  the  effectiveness  of  the  staff 
in  coordinating  the  fire  engagements  of  the  unit.  Unless  the  unit  is  being  constrained  by  external 
factors,  such  as  when  the  OPFOR  has  been  detected  massing  along  an  international  border  but 
hostilities  have  not  yet  broken  out  or  when  ammunition  supplies  are  limited,  the  shorter  the 
difference  in  time  between  when  a  target  is  detected  and  then  subsequently  killed,  the  better  the 
staff  may  be  in  coordinating  fires. 

The  Trial  1  data  indicate  that  the  unit  was  able  to  kill  the  majority  of  OPFOR  targets  within 
120  minutes  of  the  time  they  were  detected.  During  FCC  ,  resupply  rates  for  key  ammunition 
types  were  constrained  which  may  explain  why  very  few  targets  were  killed  within  30  minutes  of 
detection  or  many  targets  were  killed  more  than  1 80  minutes  after  they  were  detected.  Results 
from  additional  trials  would  be  needed  to  establish  a  baseline  to  determine  if  the  performance 
indicated  by  this  trial  should  be  sustained  or  improved  upon. 

The  picture  (Figure  20)  is  derived  from  plotting  the  location  of  the  OPFOR  target  when  it 
was  detected  and  the  location  of  where  it  was  killed.  The  lines  on  the  picture  simply  link  the  two 
points  and  do  not  indicate  the  route  that  the  target  took  before  it  was  killed.  The  unit’s  zone  of 
action  is  also  plotted  from  a  table  created  by  an  observer.  An  arrow  is  used  to  indicate  the 
OPFOR  general  direction  of  movement.  Figure  20  complements  the  data  presented  in  Figure  19. 
Based  on  ground  tactical  movement  speeds  of  25  kilometers  per  hour  using  highways,  the 
clustering  of  dead  vehicle  symbols  in  the  center  of  the  zone  of  action  indicate  that  the  majority  of 
vehieles  did  not  move  between  the  time  they  were  detected  and  the  time  they  were  attacked. 
Vehicles  that  were  detected  outside  of  unit’s  zone  of  action  were  able  to  travel  greater  distances 
before  they  were  killed,  which  would  result  in  an  increase  in  the  sensor-shooter  lag  time  for 
those  engagements. 
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Figure  19.  Sensor-shooter  time  lag  graph. 
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Note.  OPFOR  =  opposing  forces. 


Figure  20.  Sensor- shooter  time  lag  picture. 

There  are  several  potential  improvements  that  could  be  made  to  the  output  formats.  If 
objective  standards  have  been  established  for  the  time  lag  between  when  sensors  have  detected  a 
target  and  the  time  it  was  killed,  the  bars  outside  of  the  tolerance  could  be  given  a  different  color 
or  pattern  to  distinguish  how  much  unit  needs  to  sustain  or  improve  its  targeting  performance. 
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Time  of  detection  and  time  of  kill  could  be  added  to  the  picture  to  give  a  representation  of  the 
time  lag  for  individual  targets.  This  would  be  most  beneficial  for  a  few  critical  targets  since 
providing  that  information  for  all  targets  would  clutter  the  presentation,  particularly  where  there 
are  large  numbers  of  vehicles  clustered. 


Decision-Making 


Sensor  Coverage 

Figure  21  provides  information  about  the  effectiveness  of  the  various  sensors  that  were 
controlled  by  the  unit’s  staff  during  Trial  1 .  The  OPFOR  data  come  from  two  sources.  The 
locations  for  OPFOR  systems  that  are  identified  as  “Not  Detected”  are  derived  simulation  PDUs. 
These  locations  are  available  at  5-minute  intervals.  The  locations  for  systems  that  have  been 
identified  as  “Detected”  are  provided  by  data  that  record  the  location  and  time  at  which  the 
OPFOR  vehicle  is  under  observation  and  the  identification  of  the  unit  or  vehicle  that  is  observing 
it.  The  data  have  been  filtered  so  that  only  those  OPFOR  vehicles  that  were  first  detected  by  and 
are  currently  being  observed  by  the  unit  are  displayed.  These  data  are  available  in  20-minute 
intervals.  The  detected  OPFOR  vehicle  data  are  filtered  again  to  identify  those  vehicles  that 
have  a  “killed”  status  at  the  time  they  are  being  observed.  In  creating  the  picture,  the  “Not 
Detected”  systems  are  plotted  first  with  an  outlined  diamond  symbol.  The  “Detected”  vehicles 
are  plotted  next  in  a  red  diamond  symbol.  The  detected  vehicle  symbol  will  be  superimposed 
over  the  “Not  Detected”  symbol.  The  “Dead”  symbol  is  a  black  diamond  which  is  then 
superimposed  over  the  detected  symbol.  The  unit  zone  of  action  is  then  plotted  from  a  table 
created  by  an  observer  and  an  arrow  indicating  the  general  direction  of  OPFOR  movement  is 
added. 


Note.  AO  =  area  of  operations;  OPFOR  =  opposing  forces. 
Figure  21.  Sensor  coverage  at  0920. 
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The  rationale  for  this  measure  is  based  on  the  capability  of  the  system,  which  allows  the 

unit  commander  to  visualize  the  entire  “sensed”  battlefield  on  his  PVD.  The  information  that  is 
being  displayed  on  the  PVD  comes  from  a  wide  variety  of  sources,  but  the  sensors  that  are  being 
controlled  by  the  staff  provide  a  majority  of  the  OPFOR  information.  If  the  staff  is  not  properly 
deploying  and  monitoring  performance  of  their  sensors,  then  the  information  that  is  being 
displayed  to  the  commander  will  be  incomplete,  which  will  prevent  him  from  making  a  decision 
using  all  information  that  could  be  made  available  to  him. 

The  results  depicted  in  Figure  21  indicate  that  50  minutes  into  the  exercise,  a  considerable 
number  of  the  OPFOR  systems  that  were  participating  in  the  trial  had  not  yet  been  detected. 
However,  most  of  those  systems  that  had  been  detected  within  the  zone  of  action  were  killed. 

Depending  on  the  number  of  OPFOR  systems  that  have  been  detected  and  killed,  the  picture 
could  evolve  into  a  complex  presentation  of  information.  One  way  to  simplify  the  presentation 
of  information  may  be  to  create  two  different  pictures.  One  picture  would  present  the  “Not 
Detected”  and  “Detected”  OPFOR  system  locations,  while  a  second  picture  would  present  the 
“Detected”  and  “Dead”  vehicle  locations. 

Multiple  Fire  Engagements 

Table  5  lists  the  three  OPFOR  weapon  systems  that  were  attacked  after  they  were  reported 
as  being  killed.  Battle  damage  assessment  information  is  automatically  posted  as  the  SC"*  system 
is  updated  every  30  seconds.  With  this  information  available  to  everyone  in  the  unit,  instances 
where  the  target  is  fired  upon  after  it  is  destroyed  should  be  infrequent.  The  table  provides 
information  on  the  target  and  the  unit  systems  that  participated  in  the  multiple  engagements. 

One  of  the  critical  pieces  of  information  is  the  time  that  the  second  attack  took  place.  It  would 
be  possible  for  an  OPFOR  system  to  be  targeted  by  two  different  weapon  systems  during  the 
short  period  in  which  the  results  of  the  initial  engagement  had  not  yet  disseminated. 

Table  5 

Multiple  Fire  Engagements  Against  Targets  Previously  Reported  as  Destroyed 


Target 


Mark 

Target  Type 

Firer  Mark 

Firer  Type 

Ammo 

Effect  Time 

_101A12 

ELDO_VEH_BTR_90 

_2E10  VEH 

MAND_TRP_TRANS_CMD 

German_35AP 

KL 

9:02 

_101A12 

ELDO_VEH_BTR_90 

_2G410P 

VEH_ROB_PAM 

PAM 

KL 

10:46 

_119H13 

ELDO_VEH_2S_23 

_2F38P 

VEH__ROB__PAM 

PAM 

10:53 

ELD0_VEH_2S_23 

_2F19P 

VEHROBPAM 

PAM 

KL 

Il3:18 

_119H23 

ELDO_VEH_2S  23 

2F38P 

VEHROBPAM 

PAM 

KL 

;  10:54 

1]9H23 

ELDO  VEH  2S_23 

_2F33R 

VEH_ROBJDF_GUN 

MRAAS_MPERM 

KL 

|ll:08 

Figure  22  complements  the  data  presented  in  Table  5  by  depicting  the  locations  at  which  the 
three  multiple  fire  engagements  took  place.  This  information  may  be  of  use  in  analyzing  why  a 
particular  target  was  engaged  more  than  once. 
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Based  on  the  approximately  225  OPFOR  weapon  systems  that  were  engaged  and  killed 
during  the  trial  (Figure  1 1),  only  three  OPFOR  systems  were  subsequently  attacked  again  after 
they  had  been  killed.  This  result  may  indicate  that  the  unit  was  closely  tracking  OPFOR  battle 
damage  and  had  a  high  degree  of  confidence  that  the  SC'*  system  was  providing  them  with 
accurate  information. 

The  picture  output  format  may  provide  more  information  if  the  location  of  the  shooters  are 
also  displayed.  Adding  the  locations  of  other  OPFOR  weapon  systems  that  were  near  the 
targeted  systems  at  the  time  they  were  attacked  again  may  also  provide  additional  context  for 
analyzing  the  engagements.  Incorporating  the  table  into  the  picture  would  also  facilitate 
analysis. 


Figure  22.  Multiple  fire  engagements. 

Maneuver  Battle  Sets 

Figure  23  depicts  the  location  of  the  unit’s  maneuver  platoon  leader  vehicles  in  relation  to 
detected  OPFOR  weapon  systems.  The  unit  vehicle  location  data  are  available  at  5-minute 
intervals.  The  OPFOR  location  data  are  available  in  20-minute  intervals.  The  Project  Team  had 
originally  designed  this  measure  to  plot  the  locations  of  like  sets:  BLUFOR  platoon  leader 
vehicles  against  OPFOR  platoon  leader  vehicles.  However,  since  the  OPFOR  locations  are 
based  on  whether  they  had  been  detected  or  not,  relying  on  just  detected  OPFOR  platoon  leader 
vehicles  would  not  provide  sufficient  reliable  information  about  the  OPFOR  to  indicate  whether 
the  unit  had  positioned  its  subordinate  units  appropriately.  Consequently,  the  Project  Team 
decided  to  plot  the  locations  of  all  detected  OPFOR  weapon  systems,  rather  than  just  detected 
OPFOR  platoon  leader  vehicles. 
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The  rationale  for  this  measure  was  to  provide  an  indicator  of  whether  the  staff  estimates  and 
other  information  staff  members  provided  to  the  commander  allowed  him  to  make  the  right 
decision  on  positioning  his  maneuver  forces  prior  to  the  start  of  the  trial.  Based  on  the  early 
information  presented  in  Figure  23,  it  appears  that  the  unit’s  subordinate  maneuver  platoons  were 
positioned  to  block  the  detected  OPFOR  vehicles. 


Note.  BLUFOR  =  blue  forces;  OPFOR  =  opposing  forces. 


Figure  23.  Maneuver  Battle  Sets  at  0900. 

The  picture  format  for  this  measure  could  be  improved  by  changing  the  symbols  from 
vehicles  to  reflect  standard  Army  symbols  and  labels  for  friendly  forces  and  the  OPFOR  and 
increasing  their  size  somewhat.  It  would  also  be  helpful  to  be  able  to  automatically  depict 
OPFOR  units  that  are  expected  to  be  committed  against  the  unit,  but  have  not  yet  been  detected 
or  identified. 

CSS  Available  Over  Time 

Figure  24  depicts  the  number  of  rounds  available  for  four  critical  types  of  ammunition  over 
the  course  of  Trial  1.  The  intent  of  this  measure  was  to  determine  if  the  staff  had  adequately 
planned  to  ensure  that  the  unit  would  not  run  out  of  ammunition  or  fuel  during  a  critical  phase  of 
the  operation.  Since  fuel  was  not  a  major  factor  in  the  outcome  of  any  trial  during  FCC^,  the 
Project  Team  focused  on  ammunition  as  an  indicator  of  the  staffs  effectiveness  in  logistical 
planning  to  support  the  commander’s  intent.  The  results  indicate  that  with  the  exception  of  a  40- 
minute  period,  starting  at  2  hours  into  the  mission,  the  number  of  rounds  available  in  the  unit  was 
fairly  constant  and  by  the  end  of  the  mission  was  very  close  to  what  they  had  available  at  the 
start. 
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Time  (In  Minutes)  Since  Mission  Start 

r~  ■-»-MRAAS„CARGO  H»-MRAAS_KE  -»-MRAAS_MPERM  -»-QUICKL00k1 

Figure  24.  Ammunition  available  over  time. 

While  consideration  was  given  to  trying  to  add  a  line  to  indicate  what  the  minimum  desired 
stockage  level,  or  unit  basic  load,  was  for  these  types  of  ammunition,  the  Project  Team  was 
unable  to  automatically  generate  a  line  or  other  device  compatible  with  the  rest  of  the  data  that 
would  provide  that  information.  Such  information  would  be  useful  in  determining  whether  the 
staff  needed  to  sustain  or  improve  its  performance  in  this  area.  Changing  the  format  to  graph  the 
percentage  of  the  unit  basic  load  for  each  type  of  ammunition  being  tracked  that  is  available  over 
time  would  be  a  technique  for  overcoming  the  limitation  of  the  current  output  format.  Visibility 
on  the  number  of  rounds  would  be  lost,  however.  In  some  instances  such  as  when  the  unit  is 
cross-leveling  ammunition,  the  actual  number  of  rounds  available  would  be  an  important 
management  tool.  Further  research  is  required  to  determine  the  optimal  format  for  this  measure. 

Performance  Monitoring  and  Feedback 


Effects  of  Targeting 

Three  output  formats  were  developed  by  the  Project  Team  for  this  measure:  a  table,  a 
picture,  and  a  line  graph.  In  the  table,  the  categorization  of  HPT  and  the  number  of  expected 
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targets  for  each  category  is  manually  entered  into  the  table  prior  to  the  start  of  the  trial  by  an 
observer.  The  number  of  expected  targets  reflects  an  intelligence  estimate  by  the  unit’s  higher 
headquarters  of  OPFOR  weapon  systems  that  could  be  committed  in  the  unit’s  zone  of  action. 
The  targeted  number  is  generated  automatically  by  categorizing  the  targets  that  the  experimental 
unit  engaged  and  then  counting  the  number  in  eaeh  category.  The  number  of  targets  engaged 
could  be  higher  than  the  number  expected  if  the  number  of  OPFOR  participating  in  the  trial  is 
increased  or  if  the  unit  engages  targets  outside  its  zone  of  action.  The  picture  depicts  the  location 
of  where  each  OPFOR  vehicle  or  weapon  system  was  killed.  The  line  graph  provides 
information  similar  to  the  Damage  to  BLUFOR/OPFOR  Systems  measure  discussed  earlier.  The 
major  difference  is  that  the  Damage  measure  begins  counting  the  damage  from  a  zero  point 
while  this  measure  starts  with  the  expected  number  and  counts  down.  Another  difference  is  that 
the  Damage  measure  provides  the  data  in  percentages,  while  numbers  are  used  for  this  measure. 

The  rationale  for  this  measure  is  that  destruction  of  HVTs/HPTs  by  the  weapon  systems  that 
are  under  the  control  of  the  experimental  unit’s  staff  may  be  an  indicator  of  the  staffs  ability  to 
effectively  monitor  their  unit’s  performance  against  the  desired  result  and  to  provide  necessary 
feedback  to  the  commander  or  guidance  to  subordinate  units  to  get  back  on  track. 

The  results  for  the  measure  provided  in  Table  6  indicate  that  the  number  of  Tank_FVs  and 
Air  Defense  systems  targeted  were  somewhat  greater  than  the  number  expected.  This  difference 
can  be  seen  in  the  picture  (Figure  25  )  which  depicts  the  location  of  the  targets  that  were 
destroyed.  Several  targets  are  located  outside  of  the  unit’s  zone  of  action,  which  may  explain 
why  the  number  of  Tank_FVs  targeted  exceeded  the  expected  number. 

Table  6 


Effects  of  Targeting 


High  Payoff  Target 

Expected 

Targeted 

Tank_FV 

40 

53 

IFV_APC 

153 

132 

Artillery 

90 

20 

Air  Defense 

17 

22 

Note.  FV  =  fighting  vehicle;  IFV_APC  =  infantry  fighting  vehicle/armored  personnel  carrier. 
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Figure  25.  Effects  of  targeting. 

The  results  from  the  line  graph  (Figure  26)  indicate  that  the  unit  was  able  to  target  and 
destroy  in  a  short  period  of  time  almost  all  of  the  HVT/HPT  except  for  artillery  systems.  The 
effects  of  this  targeting  effort  can  be  also  be  seen  in  the  results  of  from  the  Damage  Measure 
(Figure  1 1)  where  it  shows  that  the  unit  suffered  significant  losses  from  OPFOR  artillery  systems 
with  minimum  losses  from  other  types  of  OPFOR  weapon  systems. 
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Time  Since  Mission  Stat  (Hours:Minutes) 


Tank_FV  "*»^1FV_APC  *^^Ar1illery 


Figure  26.  Effect  of  targeting  opposing  force  high  value  targets  over  time. 


The  table  format  could  be  improved  by  breaking  down  the  categories  of  high  pay  targets 
into  individual  weapon  system  types.  This  might  help  the  staff  look  to  sustain  or  sustain  their 
targeting  efforts  against  particular  systems,  not  just  against  categories  of  weapons.  Also,  the 
killed  symbols  on  the  picture  could  be  coded  by  color  or  pattern  to  reflect  the  categorization  of 
weapon  systems. 


Shared  Situational  Awareness 


Map  Area 

Figure  27  displays  the  average  area  of  terrain  that  was  visible  on  the  electronic  map  displays 
of  commanders  and  staff  officers  over  a  designated  time  interval  of  5  minutes.  The  Project  Team 
selected  the  unit  commander,  the  deputy  commander,  and  the  two  control  node  officers  in  charge 
(Outlaw6  and  Headhunter6)  as  representative  of  the  amount  of  the  battlefield  that  the  staff  would 
be  viewing  at  critical  points  during  the  mission.  The  Project  Team  also  decided  to  depict  the 
battlefield  area  that  the  four  maneuver  team  commanders  displayed  during  the  same  interval  as 
an  indicator  of  where  significant  combat  activity  would  be  occurring.  The  size  of  the  area  being 
displayed  has  been  averaged  for  the  same  5-minute  interval  for  each  of  the  designated 
commanders  and  staff  officers.  This  approach  was  necessitated  because  of  the  difficulty  in 
synchronizing  the  displays  to  the  same  instant  for  the  eight  soldiers  being  tracked.  For  example, 
over  the  7  1 /2-hour  duration  of  Trial  1,  the  experimental  unit  commander  resized  and/or  changed 
the  center  point  of  his  display  905  times.  The  deputy  commander  changed  his  display  1,363 
times  while  one  team  commander  changed  his  display  2,664  times.  Other  information  being 
displayed  includes  the  unit  zone  of  action,  the  location  of  the  unit’s  subordinate  maneuver 
platoon  leader  vehicles,  and  the  location  of  detected  OPFOR  systems.  All  data  elements  are 
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based  on  5-minute  intervals,  except  for  the  location  of  the  OPFOR  which  is  available  in  20- 
minute  increments.  An  arrow  indicating  the  general  direction  of  movement  for  the  OPFOR  has 
also  been  added. 

The  rationale  for  this  measure  is  to  ascertain  if  the  staff  has  a  shared  understanding  of  what 
is  occurring  at  critical  points  during  the  mission.  If  the  team  commanders  are  focused  on 
different  portions  of  the  battlefield  than  the  staff  or  if  the  resolution  of  the  staffs  view  is 
significantly  different  than  that  of  the  team  commanders’,  then  the  amount  of  shared  situational 
awareness  among  the  unit  comes  into  question. 

The  measure  results  depicted  in  Figure  27  indicate  that  at  0920  or  about  an  hour  into  the 
mission,  the  unit  commander  and  his  staff  had  adjusted  their  displays  to  cover  most  of  the  unit 
zone  of  action  and  could  also  cover  a  significant  portion  of  the  battlefield  in  the  area  from  which 
the  OPFOR  could  be  expected  to  appear.  Only  one  staff  officer  (Outlaw6)  was  able  to  see  back 
to  areas  where  the  unit  still  had  forces.  The  team  commanders  (Eagle6,  Fox6  Ghost6,  and 
Hawk6)  had  zoomed  their  displays  down  into  the  immediate  fight,  foeusing  on  their  subordinates 
and  the  OPFOR. 

Plotting  the  map  area  for  the  commander  and  three  staff  officers  plus  the  four  team 
commanders  on  the  same  picture  creates  a  fairly  complex  presentation.  While  it  is  not  possible 
to  replicate  it  on  paper,  the  electronic  version  of  the  output  format  can  be  modified  to  remove 
unwanted  data,  which  can  simplify  the  presentation.  Additionally,  each  of  the  individual  map 
areas  can  be  also  be  highlighted  if  needed. 
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Note.  AO  =  area  of  operations;  BLUFOR  =  blue  forces;  Cdr  =  commander; 
DepCdr  =  deputy  commander;  OPFOR  =  opposing  forces. 


Figure  27.  Map  area  at  0920. 

Surprise  Attack 

There  are  two  output  formats  associated  with  this  measure  -  a  picture  and  a  pie  chart.  The 
Project  Team  used  the  range  and  the  angle  of  attack  based  on  the  center  line  of  individual  combat 
systems  to  determine  what  constituted  a  surprise  attack.  If  the  engagement  was  less  than  4,000 
meters  and  the  individual  vehicle  was  attacked  from  the  side  or  rear,  then  the  attack  was 
considered  to  be  a  flank  engagement  and  thus  a  surprise  attack.  [Normally,  if  the  targeted 
vehicle  is  aware  of  the  location  of  its  attacker,  it  orients  its  front,  where  its  armor  is  thickest,  to 
the  threat.  If  it  is  attacked  from  the  flank  or  rear,  it  is  likely  that  the  vehicle  did  not  know  that  it 
was  vulnerable  to  attack.]  The  picture  (Figure  28)  represents  the  locations  where  BLUFOR  and 
OPFOR  targeted  vehicles  were  attacked  from  the  flank  or  rear.  The  two  graphs  (Figures  29  and 
30)  depict  the  relationship  of  flank  engagements  to  non-flank  engagements  for  both  sides.  The 
solid  pattern  indicates  the  non-flank  engagements;  the  diamond  pattern  indicates  the  flank 
pattern.  Notes  have  been  added  by  the  authors. 

The  rationale  for  this  measure  was  that  if  the  staff  was  aware  of  the  location  of  the  OPFOR, 
it  would  ensure  that  everyone  in  the  unit  was  aware  of  the  situation  and  direct  the  unit’s 
maneuver  to  either  meet  the  threat  head  on  or  to  a  position  where  the  OPFOR  could  be  attacked 
from  the  flank  or  rear,  increasing  the  probability  of  destroying  the  OPFOR  while  reducing  the 
risk  to  the  unit. 
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The  results  may  indicate  that,  even  with  the  inherent  capabilities  of  the  SC"*  system  to 
provide  a  visual  display  of  all  friendly  and  detected  OPFOR  weapon  systems  locations  that  is 
being  constantly  updated,  the  closer  the  forces  move  toward  each  other,  the  less  valuable  that 
information  is  in  protecting  the  force.  For  each  side,  the  losses  are  surrounded  by  losses  from  the 
other  side.  A  trained  observer  may  conclude  that  the  two  forces  had  intermingled  and/or 
bypassed  each  other  which  created  or  increased  the  vulnerability  to  flank  or  rear  engagements. 


Figure  28.  Surprise  attack. 

Figure  29  graphs  the  flank  or  rear  engagements  as  a  percentage  of  the  total  engagements 
against  the  BLUFOR  experimental  unit.  Twelve  percent  of  the  engagements  against  the  unit 
were  from  the  flank  or  rear.  Additional  analysis  of  these  engagements  during  an  AAR  by  the 
unit  participants  may  provide  the  reason  for  these  losses.  Figure  30  provides  the  same 
information  about  engagements  against  the  OPFOR.  While  the  total  number  losses  for  the 
OPFOR  was  greater  than  the  unit’s  losses,  the  percentage  difference  is  not  nearly  as  great. 
Again,  additional  analysis  is  required  to  explain  why  the  unit  was  not  more  effective  in 
maneuvering  forces  or  fires  once  the  OPFOR  had  closed  to  within  4,000  meters  of  their 
locations. 
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Figure  29.  Opposing  force  (OPFOR)  flank  engagements  against  blue  forces. 

While  the  picture  format  provides  a  good  representation  of  where  the  surprise  attacks  took 
place,  information  about  the  location  of  the  shooter  and  whether  it  had  been  detected  or  not 
(thereby  having  its  location  displayed  on  the  SC'*  system  PVD)  might  be  useful  in  determining  if 
the  unit  was  reacting  quickly  enough  to  an  imminent  threat  to  prevent  losses.  Including  the  total 
number  of  engagements  on  the  graphs  would  make  them  more  meaningful.  Additionally,  they 
should  be  combined  to  allow  side  by  side  comparisons. 


Figure  30.  Blue  forces  (BLUFOR)  flank  engagements  against  opposing  forces. 
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Fire  Support  Coordination 

The  Project  Team  developed  two  output  formats  for  this  measure:  a  picture  and  a  bar  graph. 
The  picture  (Figure  3 1)  depicts  the  location  where  each  individual  OPFOR  weapon  system  was 
killed  by  the  experimental  unit  during  Trial  1.  The  outlined  diamond  symbol  represents  the 
location  where  vehicles  were  destroyed  by  direct  fire  weapons.  The  solid  diamond  symbol 
represents  the  location  where  OPFOR  were  killed  by  indirect  fire  weapons.  The  unit  zone  of 
action  has  been  plotted  based  on  a  table  created  by  an  observer  and  an  arrow  indicating  the 
general  direction  of  movement  for  the  OPFOR  has  been  added.  The  bar  graph  (Figure  32  relates 
the  number  of  direct  fire  kills  to  the  number  of  indirect  fire  kills. 

The  rationale  for  this  measure  is  that  if  the  staff  has  sufficient  situational  awareness  as  well 
as  the  ability  to  synchronize  the  fire  support  assets  under  its  control,  then  the  unit  could  inflict 
significant  damage  to  the  OPFOR  before  the  enemy  can  close  to  within  direct  fire  range.  The 
farther  away  from  the  unit  the  OPFOR  can  be  destroyed,  the  less  risk  there  would  be  to  the 
experimental  unit. 


Figure  3 1 .  Fire  support  coordination. 

As  can  be  seen  in  Figure  3 1 ,  a  majority  of  the  OPFOR  were  killed  with  indirect  fire  weapon 
systems,  with  most  of  these  occurring  with  the  zone  of  action.  There  were  some  kills  outside  of 
the  zone  of  action  which  means  that  the  unit  was  able  to  interdict  the  movement  of  the  OPFOR. 
Figure  32  relates  the  number  of  BLUFOR  direct  fire  kills  to  the  number  of  indirect  fire  kills. 

The  data  indicate  that  there  were  approximately  three  indirect  fire  kills  for  every  one  direct  fire 
kill,  which  may  indicate  that  the  experimental  unit  was  able  to  inflict  significant  damage  to  the 
OPFOR  while  minimizing  its  losses. 
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Figure  32.  Blue  forces  fire  support  coordination  graph. 

Differentiating  among  the  types  of  fires  may  improve  the  picture  format  for  this  measure. 
For  example,  each  kill  for  a  specific  type  of  weapon  might  be  given  a  different  color  or  pattern. 
Another  potential  improvement  might  be  to  incorporate  the  bar  graph  into  the  picture  to  provide 
a  quick  visual  ratio  of  direct  fire  to  indirect  fire  kills  without  having  to  display  the  two  formats 
simultaneously. 


Air  Defense  Coverage 

Prior  to  the  start  of  FCC^,  the  Project  Team  surmised  the  12  indirect  weapon  systems  (Long- 
range  Attack  Missile  [LAM],  Rocket,  and  155mm  Howitzer)  controlled  by  the  staff  would  be  the 
critical  assets  most  vulnerable  to  air  attack.  In  Figure  33,  these  systems  are  represented  by  the 
triangular  symbols.  To  establish  if  the  unit’s  air  defense  systems  could  protect  these  critical 
assets,  the  locations  of  the  six  individual  air  defense  weapon  systems  were  entered  into  a 
worksheet  which  automatically  calculated  the  range  template  for  each  weapon  system.  The 
range  templates  were  then  plotted.  Since  geographical  information,  such  as  mountains,  was  not 
being  factored  into  the  display,  a  note  was  added  to  remind  the  viewer  that  the  masking  effect  of 
terrain  was  not  being  considered.  This  meant  that  the  actual  range  template  could  in  fact  be  more 
limited  than  shown.  The  direction  from  which  potential  air  attacks  could  come  was  indicated  by 
an  arrow  representing  the  general  direction  of  OPFOR  movement.  The  vehicle  location  data 
used  to  create  the  picture  is  available  in  5-minute  intervals. 

During  FCC^,  the  unit’s  air  defense  systems  came  under  the  direct  control  of  the  staff  The 
Project  Team  projected  that  the  location  of  the  air  defense  systems  in  relation  to  critical  assets 
may  be  an  indicator  of  the  staffs  ability  to  monitor  the  location  of  those  assets  and  to  direct 
changes  in  locations  based  on  the  commander’s  priorities.  With  the  relatively  short  range  (5,000 
meters),  the  air  defense  systems  would  need  to  be  positioned  very  closely  to  the  assets  and  to  be 
mutually  supporting  if  they  were  to  provide  a  defense  in  depth  against  air  attack. 
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Figure  33.  Air  defense  coverage  at  0900. 

Fratricide/Collateral  Damage 

This  measure  looks  at  the  instances  where  the  unit  attacked  and  destroyed  either  friendly 
combat  forces  (fratricides)  or  caused  unintended  damage  to  neutrals,  non-combatants,  and 
civilians  (collateral  damage).  In  addition  to  noting  the  time,  the  unit  involved  in  the  incident,  a 
description  of  the  target,  and  its  location,  the  weapon  system  and  ammunition  employed  along 
with  the  range  are  also  tracked. 
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If  the  staff  is  keeping  the  rest  of  the  unit  apprised  of  non-combatant  locations  and  other 
friendly  forces  locations  and  activities,  there  should  be  no  instances  of  fratricide  and  very  few 
instances  of  collateral  damage  caused  by  units  equipped  with  advanced  O'*!  systems  like  the  SC'* 
that  provide  users  with  updated  situational  information  every  60  seconds  or  less.  Fratricide 
and/or  collateral  damage  may  indicate  problems  with  situational  awareness  within  a  unit.  The 
type  of  ammunition  employed  and  the  range  may  indicate  if  the  unit  used  area-type  or  “dumb” 
ammunition  against  targets  that  should  have  been  engaged  with  point-type  or  “smart” 
ammunition.  Table  7  lists  the  fratricides  and  collateral  damage  that  occurred  during  Trial  1 . 

Table  7 


Fratricides  and  Collateral  Damage 


Time 

Unit 

System 

Ammo 

Range 

Target 

9:53 

_2E48R 

VEH^ROBPAM 

PAM 

2988 

FRANC_VEH_LT_TRK 

10:41 

_2G110P 

VEH_ROB_PAM 

PAM 

30994 

CIV_DI_QUAD_E 

11:31 

_2G20 

VEPI_MAND_TRP_TRANS_ 

_CMD 

German_35AP 

880 

ClV_DI_DUO_A 

13:27 

_2G20 

VEH__MAND_TRP_TRANS_ 

CMD 

M792 

112 

CtV_DI_COL20_C 

The  collateral  damage  reported  requires  further  analysis.  As  an  artifact  of  the  virtual 
simulation  used  to  support  experimentation  during  FCC^,  certain  OPFOR  units  were  disguised  as 
civilians  or  non-combatants.  This  labeling  allowed  those  units  to  approach  elements  of  the 
experimental  unit  without  drawing  undue  attention  to  themselves.  On  command,  they  would 
then  attack  the  experimental  unit  without  warning.  Further  analysis  may  determine  if  this  was 
the  case  during  this  trial. 

Figure  34  shows  the  location  of  the  collateral  damage  incidents  and  the  locations  of  the 
shooters  who  initiated  the  attacks  during  Trial  1 .  The  large  rectangles  are  the  shooters.  The 
smaller  rectangles  are  the  targets.  Lines  are  used  to  pair  shooters  with  targets.  In  two  instanees, 
the  shooter  and  target  were  so  elose  together  that  the  line  between  them  was  merged  into  the 
location  symbols.  The  legend  was  eliminated  from  this  picture.  The  software  code  that 
generates  this  picture  was  sized  to  accommodate  50  potential  collateral  damage  incidents.  Every 
potential  incident  is  automatically  assigned  a  “Series”  label  which  is  displayed  in  the  legend 
along  with  a  label  for  an  actual  incident.  The  Project  Team  determined  that  this  information 
would  create  confusion  about  the  number  of  incidents  being  displayed. 
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Figure  34.  Collateral  damage. 

Figure  35  provides  the  same  data  for  fratricides  during  Trial  1.  Again  the  legend  was 
eliminated.  There  was  only  one  instance  of  fratricide:  a  friendly  nation’s  light  truck  was 
destroyed  by  a  PAM.  Again  the  location  of  the  shooter  is  indicated  by  the  larger  of  the 
rectangles;  the  target  location  is  the  smaller  rectangle.  The  line  that  connects  the  pair  has  merged 
into  the  symbols.  Without  other  data  to  explain  what  precipitated  the  attack,  this  incident  would 
be  analyzed  during  the  unit’s  AAR  to  see  if  tactics  or  procedures  needed  to  be  modified  to 
prevent  a  recurrence. 

Initially,  the  Project  Team  sought  to  combine  the  fratricides  and  collateral  damage  incidents 
into  one  picture.  This  combination  was  not  technically  feasible  during  this  effort.  The  specific 
problem  was  linking  two  different  sets  of  shooter-target  combinations,  yet  differentiating 
between  the  two.  Additional  programming  should  be  able  to  overcome  this  challenge. 
Additionally,  adding  the  data  table  information  into  the  picture  would  also  be  useful.  To  gain  a 
full  understanding  of  the  measure’s  results  as  it  is  currently  formatted,  both  the  picture  and  the 
table  need  to  be  displayed  simultaneously. 
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Figure  35.  Fratricides. 

Common  Picture 

To  develop  this  measure  (see  Figure  36),  the  Project  Team  first  obtained  the  location  of 
every  vehicle  in  the  unit.  This  location  is  plotted  using  a  red  or  blue  outlined  diamond  or  square. 
The  vehicle  was  assigned  the  “Stationary”  label.  If  there  were  not  stationary  vehicles,  such  as 
the  case  for  the  OPFOR,  then  Excel  assigned  the  “Series”  label.  The  location  of  the  same 
vehicle  20  minutes  later  was  then  plotted.  The  20-minute  interval  was  used  for  two  reasons: 
first,  the  Project  Team  thought  that  vehicle  movements  of  less  than  20  minutes  in  duration  would 
not  be  visually  detectable  at  the  scale  of  terrain  data  being  presented;  and  second,  OPFOR 
information  was  only  available  in  20-minute  intervals.  This  location  was  plotted  in  a  solid  red  or 
blue  diamond  or  square,  and  labeled  as  “New/Moved.”  This  was  the  same  category  used  to 
account  for  vehicles  that  may  have  been  added  to  either  the  OPFOR  or  the  unit  since  vehicle 
positions  had  been  last  recorded.  If  a  vehicle  was  being  reported  as  destroyed,  it  was  plotted  in  a 
black  diamond  or  square,  and  labeled  “Dead.” 

An  indicator  of  how  much  information  the  staff  and  others  in  the  unit  are  sharing  among 
themselves  would  be  an  analysis  of  the  changes  over  time  in  the  information  that  is  being 
displayed  on  an  individual  soldier’s  PVD.  This  information  would  be  a  common  starting  point 
for  most  staff  actions  such  as  providing  estimates,  making  recommendations,  alerting  the 
commander  to  changes  in  the  situation,  or  developing  courses  of  action.  To  start  that  analysis, 
the  first  step  was  to  determine  what  information  changes.  The  Project  Team  focused  on  the 
locations  and  status  of  detected  OPFOR  and  unit  vehicles  since  changes  in  locations  and  status 
would  be  readily  discernible  on  a  PVD. 
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The  results  obtained  by  this  measure,  at  the  scale  being  represented,  required  careful 
examination  to  detect  significant  changes,  even  at  the  20-minute  interval  used  to  plot  vehicle 
locations.  Any  movement,  however  insignificant,  caused  the  vehicle  to  be  plotted  as 
“New/Moved.”  Vehicles  that  were  “Dead”  continued  to  be  plotted  as  each  picture  was  being 
updated,  so  that  the  vehicles  that  had  been  killed  during  the  current  20-minute  interval  could  not 
be  discerned  from  those  killed  in  the  preceding  20-minute  interval  or  even  earlier. 
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Figure  36.  Common  picture  at  0920. 

To  fully  realize  the  intent  of  this  measure,  additional  information  is  required  about  the 
electronic  PVD  filters  each  FCC^  unit  participant  had  set  on  his  system  at  the  point  in  time  that 
the  picture  was  created.  For  example,  a  participant  may  have  set  the  PVD  to  display  aggregated 
unit  symbols  such  as  platoons  or  companies  rather  than  individual  vehicles.  Rather  than  having 
hundreds  of  vehicle  symbols,  his  display  may  have  had  only  10  or  15  unit  symbols.  He  could 
have  adjusted  the  display  so  that  dead  or  killed  vehicle  symbols  would  not  be  displayed.  He 
could  have  adjusted  the  display  to  remove  or  to  leave  OPFOR  vehicle  symbols  that  were  not 
currently  being  observed  by  some  sensor.  The  Project  Team  was  not  able,  within  the  time 
available  to  the  project,  to  integrate  individual  participant  filter  settings  into  the  picture. 
Additionally,  the  amount  of  vehicle  movement  that  would  be  considered  significant  needs  to  be 
defined,  so  that  only  those  significant  movement  changes  would  be  plotted.  Also,  newly  killed 
vehicles  need  to  be  distinguished  from  previously  killed  vehicles. 
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Conclusions 


During  FCC^,  the  Project  Team  implemented  20  measures,  19  of  which  were  automated  in 
that,  once  an  initial  set  of  criteria  had  been  determined  (such  as  the  critical  weapon  systems  to 
track,  the  number  of  unit  and  OPFOR  weapons  participating  in  the  exercise,  and  unit  zones  of 
action),  the  output  formats  could  be  generated  from  trial  data  without  further  observer 
intervention.  The  orders  distribution  measure  requires  additional  development  to  automate  it. 
Further  refinement  is  required  to  improve  the  utility  of  some  of  the  measures  as  discussed  under 
the  particular  measures. 


Lessons  Learned:  Improve 

This  project  built  on  the  lessons  learned  from  the  previous  projects  and  more  fully  realized 
the  development  of  fully  automated  measures  of  command  and  staff  performance.  The 
implementation  of  the  automated  measures  during  FCC^  provided  an  opportunity  to  test 
generating  automated  measures  output  formats  in  training  conditions  similar  to  those  that  are 
envisioned  for  future  brigade-level  and  below  battle  staffs  equipped  with  advanced  digital 
systems.  Flowever,  there  are  still  issues  that  need  to  be  addressed  before  these  automated 
measures  can  become  a  meaningful  part  of  training  feedback  in  an  AAR-type  environment. 

Observer  Input 

A  key  role  for  an  observer  in  any  training  event  is  determine  before  hand  what  information 
will  be  needed  to  provide  meaningful  performance  feedback.  Even  training  feedback  systems 
that  rely  heavily  on  automated  measures  need  someone  to  determine  which  measures  from  the 
available  library  will  apply  to  the  specific  event  and  to  review  the  outputs  to  determine  if  the 
results  should  be  incorporated  into  an  AAR.  Observations  from  trained  observers  are  also 
required  to  put  many  of  the  measure  outputs  into  context.  All  of  the  conditions  involved  with 
staff  performance  during  a  training  event  cannot  be  automatically  derived.  For  example,  while 
the  amount  of  map  area  that  is  being  displayed  on  a  staff  member’s  PVD  can  be  determined,  the 
data  cannot  reveal  whether  the  staff  member  was  actually  looking  at  the  display  or  attending  to 
other  duties,  which  may  have  caused  him  to  ignore  the  information  being  displayed.  A  staff 
member  simply  could  have  left  the  display  on  and  exited  his  vehicle.  Also,  some  measures,  such 
as  fratricide  and  collateral  damage,  invariably  will  require  a  detailed  examination  of  all  the 
circumstances  involved  with  those  incidents.  The  measure  output  alerts  the  training  participants 
that  a  problem  exists,  but  does  not  get  to  the  root  cause.  Without  objective  command  and  staff 
performance  standards,  all  automated  measures  will  need  to  be  interpreted  by  SMEs  who  can 
provide  standards  based  on  their  knowledge  and  experience. 

Data  File  Size 

After  researching  the  capabilities  in  Microsoft®  Excel,  the  Project  Team  realized  that  the 
quantity  of  records  that  would  be  produced  during  an  FCC^  trial  could  not  be  manipulated  in 
Excel  as  planned.  Recording  1,000  vehicle  positions  every  five  minutes  for  an  eight  hour  trial 
produced  96,000  records  which  exceeded  Excel’s  data  handling  capabilities.  This  caused  the 
Project  Team  to  go  from  Excel  to  Microsoft®  Access.  Adding  an  extra  software  package  into  the 
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equation  led  to  some  minor  difficulties  in  the  design  of  the  observer  workstation.  Instead  of 
pushing  a  single  button  in  order  to  develop  a  measure  as  planned,  the  user  has  to  push  a  button  in 
each  software  package.  This  leads  to  redundancy  for  the  user  as  well  as  longer  measure  output 
development  time.  Additionally,  more  data  tables  were  required  than  were  planned.  For 
example,  the  largest  MMBL  ASCII  data  file  was  eventually  split  into  three  different  files  so  the 
information  needed  to  build  measures  could  be  accessed  more  easily.  Additionally,  the  sheer 
size  of  the  files  associated  with  FCC^  Access  databases  in  excess  of  51  Megabytes  and  some 
Excel  output  files  exceeding  three  Megabytes,  created  problems  in  transferring,  copying,  and 
printing  operations.  All  of  these  challenges  in  handling  data  are  essentially  technology  driven 
and  can  be  overcome  with  more  capable  computer  and  communications  hardware  and  software. 

Missing  Data 

Working  with  the  initial  set  of  FCC^  data,  the  Project  Team  discovered  that  there  were 
numerous  queries  that  were  producing  files  with  no  data.  This  condition  was  not  expected  since 
it  had  not  occurred  during  pre-implementation  testing.  As  a  result  of  having  blank  query  files, 
the  Microsoft®  Excel  macros  would  generate  run-time  errors  which  terminated  further  data 
processing  necessary  to  create  the  output  formats.  To  overcome  these  errors,  the  Project  Team 
encoded  the  Excel  macros  to  ignore  empty  data  files.  The  Project  Team  also  compared  the  raw 
ASCII  data  files  with  the  FCC^  Experiment  naming  convention  for  identifying  unit,  vehicles,  and 
weapon  systems.  Several  discrepancies  were  found  which  resulted  in  the  Project  Team  changing 
queries  to  reflect  actual  conditions.  Finally,  the  Project  Team,  based  on  input  from  MMBL 
personnel,  determined  that  the  output  formats  were  not  reflecting  all  activity  that  was  occurring 
during  the  trials.  An  examination  of  the  MMBL  ASCII  data  files  revealed  that  the  file  that 
matched  vehicle  locations  with  engagements  was  not  catching  every  engagement.  This  file  was 
then  split  by  MMBL  programmers  into  two  data  files  which  corrected  the  problem.  This,  in  turn, 
led  the  Project  Team  to  revise  all  queries  requiring  engagement  data.  With  these  changes, 
problems  with  empty  data  files  were  eliminated. 

Data  Manipulation 

The  goal  for  this  project  was  to  have  the  outputs  from  the  automated  measures  to  be 
available  for  use  during  the  FCC^  AARs  which  would  be  held  each  day.  Based  on  previous 
experience,  the  Project  Team  had  projected  that  it  would  take  about  two  hours  to  produce  the 
complete  set  of  output  formats.  This  would  allow  them  to  be  used  during  FCC^  AARs,  which 
generally  took  place  about  two  hours  after  the  last  significant  combat  action  during  a  trial. 
Unfortunately,  the  amount  of  time  to  create  the  output  formats  greatly  exceeded  that  estimate.  It 
took  an  average  of  two  hours  of  processing  time  for  the  software  used  by  the  MMBL  to 
consolidate  the  data  and  produce  the  initial  data  tables  for  each  trial;  it  took  another  15-30 
minutes  to  convert  these  data  files  to  ASCII  and  post  them  to  an  MMBL  intranet  web  page  where 
the  Project  Team  could  access  them;  it  took  another  30-45  minutes  to  import  the  ASCII  files  into 
Microsoft®  Access;  and  finally,  it  took  an  additional  45-60  minutes  to  build  the  measures,  add 
titles,  and  save  the  output  foimats.  The  total  amount  of  processing  time  from  start  to  finish 
exceeded  four  hours.  As  a  result,  the  results  where  not  available  for  use  during  the  FCC^  AARs. 
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Lessons  Learned:  Sustain 


Based  on  the  experience  of  the  Project  Team  during  the  FCC^  implementation,  as  well  as  the 
discussion  for  improving  each  of  the  measures  provided  earlier  in  this  report,  several  lessons 
were  learned  which  may  serve  to  sustain  current  efforts  in  implementing  automated  measures  of 
command  and  staff  performance  and  to  act  as  a  catalyst  to  initiated  future  research.  These  may 
be  applicable  to  developers  concerned  with  embedding  training  and,  by  extension,  performance 
assessment  methodology  and  tools  into  future  digital  O'*!  systems  at  the  brigade  level  and  below. 
These  lessons  are  provided  as  issues  to  be  addressed  in  future  research. 

Data  Visualization 

Continued  efforts  are  needed  to  optimize  the  presentation  of  automated  measure  results, 
especially  for  graphical  and  pictorial  formats,  to  soldiers.  The  goal  should  continue  to  be  to 
provide  information  that  soldiers  can  readily  understand  without  facilitator  or  trainer 
intervention.  Formats  should  be  able  to  relate  command  and  staff  performance  to  battlefield 
conditions.  Users  should  be  able  to  track  or  “drill  down”  from  MOEs  (outcomes)  to  MOPs 
(processes).  Finally,  through  the  use  of  interactive  media,  soldiers  should  be  able  to  control  the 
type  and  amount  of  automatic  performance  feedback  information  that  they  are  receiving  as  well 
as  the  format  in  which  it  is  being  displayed  not  only  in  AARs  but  also  from  their  operational 
systems. 


Commercial  Business  Software 

Using  a  widely  available  commercial  off-the-shelf  business  software  package  such  as 
Microsoft®  Office  provides  inexpensive  development  tools  to  create  prototype  automated 
measures.  All  of  the  components  that  would  be  required  to  provide  measures  of  command  and 
staff  performance  to  an  automated  AAR  process  are  available:  data  storage  and  retrieval 
(Access),  charting  and  graphing  tools  (Excel),  and  tools  to  present  the  results  to  the  training 
participants  (PowerPoint®).  Applying  this  same  type  of  software  to  operational  automated 
measures  of  command  and  staff  performance  would  benefit  users  of  such  systems.  Most 
experienced  business  software  users  would  need  little  or  no  additional  training  to  operate  the 
software  and  to  work  with  the  results  from  the  measures.  Finally,  as  these  commercial  business 
software  programs  incorporate  additional  functionality,  which  may  make  them  even  easier  to 
integrate  into  O'*!  systems,  upgrades  should  incur  little  or  no  additional  training  costs. 

User  Tools  Development 

Today’s  measures,  such  as  the  prototype  automated  measures  designed  and  developed 
during  this  effort  may  not  apply  to  future  training  conditions.  There  is  a  need  for  future  staff 
trainers  to  be  able  to  design  their  own  measures  without  requiring  an  extensive  background  in 
programming.  An  automated  measures  development  package  that  gives  users  the  option  of  using 
existing  measures  or  modifying  them,  or  creating  entirely  new  measures  is  needed.  If  the  tools 
used  to  work  on  measures  development  or  modification  are  not  already  integrated  into 
operational  O'*!  systems  used  by  the  training  participants,  they  should  be  based  on  whatever 
commercial  business  software  that  is  being  widely  used  throughout  the  Army. 
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Future  Staff  Performance  Standards 


Further  development  efforts  on  automated  performance  measures  requires  research  to 
develop  objective  performance  standards  for  future  (9  organizations.  The  research  should 
include  intrinsic  measures  where  the  organization  is  provided  on-going  indicators  of  its 
performance,  such  as  Orders  Distribution,  where  they  could  determine  by  the  systems  and 
resources  immediately  available  to  them  if  everyone  is  using  the  same  operations  overlay,  and 
extrinsic  measures,  such  as  Kill  Distance,  where  additional  post-training  processing  would  be 
required  to  establish  the  organization’s  overall  performance.  Team  behaviors  and  processes, 
soldier-machine  interface,  and  soldier-machine  task  allocation  issues  should  be  included  in  the 
research  to  ensure  both  group  dynamics  and  individual  contributions  can  be  isolated  and 
analyzed  for  their  impact  to  the  overall  performance  of  the  organization. 

Pre-Planned  Measures  Implementation 

Automated  performance  measurement  capabilities  need  to  be  fully  integrated  into  the  design 
of  advanced  digital  systems.  If  data  are  not  being  recorded  and  stored,  it  is  not  available  for 

other  purposes  such  as  training  feedback.  Data  storage  may  be  an  encumbrance/obstacle  with 
operational  C'^I  systems  that  dump  data  when  they  are  powered  down.  In  such  instances, 
alternative,  dedicated  data  storage  systems  may  have  to  be  specifically  developed  for  recording 
performance  data  for  later  analysis.  The  feasibility  of  including  this  capability  may  increase  and 
the  costs  associated  with  it  may  decrease  as  large  capacity  data  storage  devices  become  more 
compact,  tolerant  of  unstable  operating  conditions,  and  power  efficient.  As  described  earlier, 
handling  large  amounts  of  data  associated  with  automated  performance  measures  can  overwhelm 
operational  9  systems.  Research  efforts  are  needed  to  optimize  collecting,  processing, 
assembling,  and  distributing  data  for  AARs. 


Summary 

Digital  C”*!  systems  provide  unlimited  potential  to  assess  individual  soldier,  small  group,  and 
collective  performance  data.  They  have  organic  capabilities  that  need  to  be  exploited  to 
automatically  collect,  analyze,  and  portray  data.  Such  data  collection,  accompanied  by 
capabilities  for  analyzing  and  displaying  the  results  in  terms  of  processes  and  outcomes,  can 
support  both  intrinsic  and  extrinsic  feedback.  By  “intrinsic  feedback,”  we  mean  the  information 
that  immediately  informs  the  user  that  something  is  not  right,  or  that  more  information  is 
available,  or  that  some  critical  information  need  is  being  answered.  This  information,  provided 
by  means  of  on-board  systems  and  remote  sensors,  ean  be  provided  as  an  operational  capability 
as  well  as  during  training.  “Extrinsic  feedback”  allows  the  user  to  look  back  on  an  operation  or 
training  exercise  and  identify  ways  to  sustain  or  improve  performance  through  optimal  use  of 
information  systems. 

Accessing  the  collected  performance  data  and  making  it  intelligible  to  observers  or  to  the 
unit  in  training  has  to  become  a  common  and  routine  feature  of  all  training  and  operational 
systems.  The  particular  data  elements  that  are  related  to  performance,  the  analytic  tools  that  fuse 
data  to  provide  useful  feedback,  and  the  format  for  feedback  reports  need  to  be  optimized  from  a 
human  factors  point  of  view  to  allow  for  clear  and  immediate  impact.  Continued  research  and 
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development  of  automated  measures  of  performance  that  provide  intrinsic  and  extrinsic  feedback 
is  an  imperative  for  training  the  Army’s  future  brigade-level  and  below  forces  equipped  with 
advanced  digital  C"*!  systems. 
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AAR 

ADA 

AO 

ARI 

ARPA 

ARSI 

ASCII 

BCR 

BCV 

BLEFR 

BLEP 

BLUFOR 

BOS 

C^ 

C^V 

C^I 

CCIR 

Cdr 

COTS 

CSS 

DA 

DC^I 

DC^I-2 

DC^I-S 

Skills 

DCA 

DepCdr 

DIS 

FCC^ 

FLOT 

FOV 

FV 

GUI 

HPT 

HVT 


Appendix  A 

List  of  Acronyms 

after  action  review 
air  defense  artillery 
area  of  operations 

U.S.  Army  Research  Institute  for  the  Behavioral  and  Social  Sciences 
Advance  Research  Projects  Agency 
ARPA  Reconfigurahle  Simulator  Initiative 
American  Standard  Code  for  Information  Interchange 

Battle  Command  Reengineering 
battle  command  vehicle 
Battle  Lab  Experiment  Final  Report 
Battle  Lab  Experiment  Plan 
blue  forces 

battlefield  operating  system 

command  and  control 
command  and  control  vehicle 

command,  control,  communications,  computers,  and  intelligence 

commander’s  critical  information  requirements 

commander 

commercial  off  the  shelf 
combat  service  support 

Department  of  the  Army 

Prototype  Methods  for  the  Design  and  Evaluation  of  Training  and 
Assessment  of  Digital  Staffs  and  Crewmen 
Refinement  of  Methods  for  the  Training  and  Assessment  of  Digital  Staffs 
Performance  Evaluation,  Training,  and  Future  Requirements  for  Digital 

Data  Collection  and  Analysis  System 
Deputy  Commander 
distributed  interactive  simulation 

Future  Combat  Command  and  Control 
forward  line  of  own  troops 
field  of  view 
fighting  vehicle 

graphical  user  interface 

high  payoff  target 
high  value  target 
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IDF 

indirect  fire 

IFV_APC 

infantry  fighting  vehicle/armored  personnel  carrier 

LAM 

long-range  attack  missile 

LOS 

line  of  sight 

MCOO 

modified  combined  obstacle  overlay 

MMBL 

Mounted  Maneuver  Battlespace  Lab 

ModSAF 

Modular  Semi-Automated  Forces 

MOE 

measures  of  effectiveness 

MOP 

measures  of  performance 

MTR 

mortar 

NCO 

noncommissioned  officer 

OPFOR 

opposing  forces 

OPORD 

operations  order 

OWS 

Observer  Work  Station 

PAM  ■ 

precision  attack  missile 

PDU 

protocol  data  unit 

PLT  LDR 

platoon  leader 

PPT 

Microsoft®  PowerPoint® 

PVD 

plan  view  display 

SAG 

SME  Advisory  Group 

SC'' 

Surrogate  Command,  Control,  Communications,  and  Computers 

SITREP 

situation  report 

SME 

Subject  Matter  Expert 

SOP 

standing  operating  procedure 

sov 

staff  operations  vehicle 

SPOTREP 

spot  report 

STAARS 

Standard  Army  After  Action  Review  System 

TRADOC 

U.S.  Army  Training  and  Doctrine  Command 

UAV 

unmanned  aerial  vehicle 

UGV 

unmanned  ground  vehicle 

VEH  ROB  MTR 

vehicle  robot  mortar 

VTC 

video  teleconference 
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Appendix  B 

Future  Combat  Command  and  Control  Experiment  Setting 

The  implementation  of  the  automated  measures  developed  was  dependent  upon  the  Future 
Combat  Command  and  Control  (FCC^)  environment.  Therefore,  descriptions  of  the  participants, 
the  equipment  used  by  the  participants,  and  equipment  used  to  collect  data  in  the  FCC^ 
experiment  follow. 


Participants 

The  unit  participating  in  the  experiment  was  an  active  Army  cavalry  squadron  staff  with  its 
subordinate  company  commanders  participating.  One  company  brought  drivers  and  gunners  to 
man  several  future  combat  vehicle  simulators.  As  shown  in  Figure  B-1,  the  squadron  staff, 
which  operated  in  a  virtual  simulation,  included  the  commander  and  13  staff  officers  and  non¬ 
commissioned  officers  (NCOs).  The  commander  and  staff  were  reconfigured  into  two  battle 
command  vehicles  (BCVs)  and  two  staff  operations  vehicles  (SOVs),  or  nodes.  The  battle 
command  reengineering  aspect  of  the  FCC^  experiment  was  focused  on  this  group  of  14  soldiers. 
The  node  functions  and  job  responsibilities  for  each  staff  member  were  left  to  the  discretion  of 
the  squadron  commander,  who  was  allowed  to  reorganize  the  staff  as  he  gained  experience  in 
operating  the  Surrogate  Command,  Control,  Communications,  and  Computers  (SC'^)  system. 
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Figure  B-1 .  Battle  Command  Reengineering  IV  staff  structure. 
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Table  B-1  shows  the  call  signs  and  associated  node  positions  for  the  commander  and  13 
primary  staff  members.  In  this  report,  these  participants  will  often  be  identified  by  their  call 
signs,  especially  in  tables  that  appear  in  the  Results  section. 

Table  B-1 


Staff  Member  Call  Signs  and  Titles 


Call  Sign 

Title 

Node 

Cougarb 

Squadron  Commander 

Command  1 

Cougar62 

Enemy  Operations  Officer 

Command  1 

Cougar69 

Effects  Officer 

Command  1 

Cougar3 

Deputy  Squadron  Commander 

Command  2 

Cougar32 

Operations  NCO 

Command  2 

Cougar35 

Operations  Officer 

Command  2 

Outlawb 

Battle  Captain 

Control  1 

Outlaw2 

Enemy  Operations  NCO 

Control  1 

Outlaw3 

Friendly  Operations  Officer 

Control  1 

Outlaws  9 

Sensor  NCO 

Control  1 

Headhunerb 

Battle  Captain 

Control  2 

Headhunter2 

Enemy  Operations  NCO 

Control  2 

Headhunters 

Friendly  Operations  Officer 

Control  2 

Headhunters  9 

Sensor  NCO 

Control  2 

Eagleb 

Company  Commander 

Constructive  Simulation 

Foxb 

Company  Commander 

Constructive  Simulation 

Ghostb 

Company  Commander 

Constructive  Simulation 

Hawkb 

Company  Commander 

Constructive  Simulation 

Note.  NCO  =  non-commissioned  officer. 


Other  squadron  personnel  involved  in  the  experiment  included:  six  company  commanders, 
six  deputy  company  commanders,  six  maneuver  platoon  leaders,  one  scout  platoon  leader,  one 
scout  platoon  sergeant,  nine  gunners,  and  15  drivers. 

Materials 

The  FCC^  used  emulation  as  well  as  constructive  and  virtual  simulators.  Figure  B-2  shows 
the  layout  of  the  SC"*  system  in  the  BCVs  and  SOVs.  The  SC'*  system  included  the  following 
capabilities: 

•  Command  and  Control  (C^)  Plan  View  Display  (PVD),  represented  by  the  Modular  Semi- 
Automated  Forces  (ModSAF)  two-dimensional  PVD.  On  the  PVD,  the  commander  and 
the  staff  are  able  to  view  movements  of  all  of  their  own  systems,  as  well  as  any  opposing 
force  (OPFOR)  units  detected  by  satellite  or  other  sensors.  Overlays  can  be  drawn  on  the 
PVD,  users  can  add  labels  or  other  notes,  and  there  are  tools  that  show  past  events  and 
project  future  movements. 
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Stealth  display,  providing  a  3 -dimensional  representation  of  the  battlefield  with  all  of  the 
systems  that  are  visible  on  the  PVD  (i.e.,  friendly  and  detected  OPFOR). 


•  Video  teleconference  (VTC)  capability  linking  the  commander  and  the  staff. 

•  Collaborative  whiteboard  capability,  to  allow  the  commander  to  present  his  intent  and 
guidance  to  the  staff  visually  and  quickly.  Users  who  are  part  of  the  whiteboard  session 
can  show  snapshots  from  their  PVDs,  draw  in  different  colors  on  those  images,  add 
clipart-style  labels  and  icons,  and  type  words  onto  the  whiteboard. 

•  Large  screen  display,  providing  a  three-dimensional  representation  of  the  battlefield  with 
all  of  the  systems  that  are  visible  on  the  PVD,  Stealth,  whiteboard,  or  unmanned  aerial 
vehicle  (UAV)  screens. 

•  Digitized  modified  combined  obstacle  overlay  (MCOO),  produced  automatically  for  the 
large  screen  display,  rather  than  as  a  manually  produced  intelligence  overlay. 

•  Satellite  imagery,  acting  as  the  electro-optic  satellite  sensor  to  deliver  a  direct  downlink 
imagery  feed. 


Note.  BLUFOR  =  blue  forces;  OPFOR  =  opposing  forces;  VTC  =  video  teleconference. 
Figure  B-2.  Surrogate  command,  control,  communications,  and  computers  system  setup. 

Vehicles  and  weapon  systems  were  represented  in  either  constructive  or  virtual  simulation. 
Constructive  simulation  (ModSAF)  was  used  to  generate  and  control  the  OPFOR,  friendly  forces 
below  the  company  level,  and  unmanned  vehicles  replicating  both  aerial  and  ground  sensors 
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(refened  to  as  UAVs  and  UGVs,  respectively).  Constructive  simulation  workstations  were  used 
by  the  Sustainment  Team  Commander,  the  Effects  Team  Commander,  and  the  four  Maneuver 
Team  Commanders.  The  remainder  of  the  extended  training  audience  was  in  virtual  simulation. 

In  the  virtual  environment,  simulators  were  used  to  represent  several  vehicles.  These 
included  the  battalion  commander  and  deputy  commander  vehicles  which  were  represented  by 
the  Advanced  Research  Projects  Agency  (ARP A)  Reconfigurable  Simulator  Initiative  (ARSI) 
simulator  and  an  ARSI  mockup,  respectively;  and  BCVs  and  SOVs  which  were  represented  by 
command  and  control  vehicle  (C^V)  mockups.  Scout  vehicles  and  the  manned  platoon  vehicles 
of  three  maneuver  teams  were  represented  by  Future  Combat  Vehicle  mockups.  The  virtual  and 
constructive  environments  were  linked  by  means  of  distributed  interactive  simulation  (DIS)  to 
form  the  seamless  battlefield  environment  for  the  participants. 

The  FCC^  experiment  trials  were  based  on  tactical  operations  that  an  Army  battalion, 
equipped  with  an  advanced  digital  command,  control,  communications,  computers,  and 
intelligence  (C'^I)  system,  might  be  expected  to  conduct  in  the  year  2010  and  beyond.  The  virtual 
terrain  chosen  for  the  experiment  was  northeastern  Bosnia-Flerzegovina,  centered  on  the  city  of 
Tuzla.  This  terrain  is  extremely  mountainous  with  limited  ground  mobility  corridors.  Figure 
B-3  shows  the  experiment  terrain  map  with  a  battalion  area  of  operations  superimposed. 


Figure  B-3.  Future  Combat  Command  and  Control  terrain  map. 


The  FCC^  experiment  data  were  collected  and  processed  by  the  Data  Collection  and 
Analysis  System  (DCA).  Figure  B-4  illustrates  key  aspects  and  functions  of  the  DCA. 
Information  about  SC'*  system  usage,  electronic  messaging,  voice  communications  (but  not 
content),  displayed  situational  awareness,  and  the  status  of  major  combat  systems  in  the 
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constructive  simulation  driving  the  experiment  were  recorded  by  three  separate  systems, 
collectively  refeiTed  to  as  the  data  logger.  Using  database  extraction  tools,  the  DCA  would 
initially  create  a  series  of  basic  data  tables.  These  tables  would  be  subsequently  refined  into 
more  advanced  tables  which  answered  specific  questions  posed  by  various  researchers.  These 
refined  tables  were  the  starting  point  for  output  formats  of  the  automated  measures  developed 
during  this  project. 


Note.  PDU  =  protocol  data  units;  SC'*  =  surrogate  command,  control, 
communications,  and  computers. 


Figure  B-4.  Future  Combat  Command  and  Control  data  collection  system. 
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Appendix  C 


Automated  Measures  of  Command  and  Staff  Performance 
Categorized  by  Team  Process  Skill  Dimensions 


Adaptability 

1 .  CSS  Locations  -  Location  of  types  of  combat  service  support  (CSS)  assets  at  a  specified  time 
period. 

a.  Operational  definition:  Location  of  CSS  supply  points  and/or  resupply  vehicles  (e.g., 
fuel,  ammunition)  in  relation  to  unit  locations  on  the  terrain  at  critical  points  during  the 
mission  (start  of  exercise,  first  indirect  fire  engagement  with  opposing  forces  [OPFOR], 
first  direct  fire  engagement  with  OPFOR,  first  friendly  casualty,  friendly  losses  exceed 
30%,  and  last  engagement  during  mission). 

b.  Rationale:  Identify  if  the  staffs  mission  planning  had  incorporated  sufficient  flexibility 
to  respond  to  unanticipated  requirements,  such  as  a  platoon  running  out  of  fuel  or 
ammunition  during  heavy  combat  or  at  a  critical  point  in  the  mission.  If  the  supply  points 
or  resupply  vehicles  were  located  where  they  were  needed  when  they  were  needed,  even 
from  an  unforeseen  requirement,  an  inference  could  be  made  that  the  staff  had  adequately 
planned. 

c.  Output:  Picture 

2.  Node  Locations  -  Location  of  each  node  in  relation  to  major  subordinate  units  at  a  specified 

time  period. 

a.  Operational  definition:  Location  of  each  node  on  the  terrain  at  critical  points  during  the 
mission  (start  of  exercise,  first  indirect  fire  engagement  with  OPFOR,  first  direct  fire 
engagement  with  OPFOR,  first  friendly  casualty,  friendly  losses  exceed  30%,  and  last 
engagement  during  mission). 

b.  Rationale:  Location  of  the  unit’s  command  and  control  (C^)  nodes  may  indicate  the 
ability  of  the  staff  to  handle  different  requirements  simultaneously  while  keeping 
positioned  to  maintain  communications  with  all  subordinate  elements,  and  maintaining 
operational  and  physical  security. 

c.  Output:  Picture 


Performance  Monitoring  and  Feedback 

1 .  Effects  of  Targeting  -  Depiction  of  high  value  targets  (HVTs)/high  payoff  targets  (HPTs) 
and  the  degree  of  attrition  each  suffered  during  a  specified  time  period. 

a.  Operational  definition:  For  targets  that  the  commander  has  designated  as  HVTs/HPTs 
(e.g.,  tanks,  artillery,  air  defense  systems,  C2  nodes),  calculate  number  of  kills 
(catastrophic,  firepower,  mobility)  over  time  and  location  and  time  of  each  target  kill. 
Also  calculate  time  first  detected,  time  engaged,  and  time  killed  for  each  target. 
Identification  of  HVTs/HPTs  will  be  provided  prior  to  start  of  exercise. 

b.  Rationale:  Destruction  of  HVTs/HPTs  by  the  weapon  systems  that  are  under  the  control 
of  the  unit’s  staff  may  be  an  indicator  of  the  staffs  ability  to  effectively  monitor  their 
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unit’s  performance  against  the  desired  result  and  to  provide  necessary  feedback  to  the 
commander  or  guidance  to  subordinate  units  to  get  back  on  track, 
c.  Output:  Picture,  line  graph,  and/or  table 


Shared  Situational  Awareness 

1 .  Map  Area  -  Square  kilometers  of  battlefield  displayed  for  each  staff  member  at  a  specified 

time  period. 

a.  Operational  definition:  Center  points  of  each  staff  personnel’s  plan  view  display  (PVD) 
screen  displayed  by  grid  coordinates  at  critical  points  during  the  mission  (start  of 
exercise,  first  indirect  fire  engagement  with  OPFOR,  first  direct  fire  engagement  with 
OPFOR,  first  friendly  casualty,  friendly  losses  exceed  30%,  and  last  engagement  during 
mission).  The  view  size  may  be  affected  by  the  size  of  the  open  PVD/SC'^  [Surrogate 
Command,  Control,  Communications,  and  Computers]  window,  the  scale  of  the  map 
seleeted,  and  the  use  of  SC"^  tools.  Pieture  formats  of  this  output  should  enable  visual 
comparison  of  visible  map  relative  to  total  map  and  relative  to  other  users. 

b.  Rationale:  May  indicate  whether  the  staffhas  a  shared  understanding  of  what  is 
occurring  at  critical  points  during  the  mission.  If  the  team  commanders  are  focused  on 
different  portions  of  the  battlefield  than  the  staff  or  if  the  resolution  of  the  staffs  view  is 
signifieantly  different  than  that  of  the  team  commanders’,  then  the  amount  of  shared 
situational  awareness  among  the  unit  eomes  into  question. 

c.  Output:  Pieture 

2.  Surprise  Attack  -  Depiction  of  blue  forces  (BLUFOR)  in  relation  to  OPFOR  when  BLUFOR 

were  attacked  for  flank  or  rear  engagements  during  each  mission. 

a.  Operational  definition:  Loeation  on  battlefield  of  flank  or  rear  direct  fire  engagements  on 
OPFOR  and  BLUFOR  vehicles:  attacks  from  a  position  that  is  greater  than  45  degrees 
and  less  than  3 1 5  degrees  of  the  hull  orientation  of  the  vehicle  being  attacked.  The 
orientation  of  the  vehicle  is  considered  to  be  zero  degrees  for  this  ealeulation.  Direct  fire 
engagements  are  those  that  occur  at  ranges  of  4,000  meters  or  less.  Calculate  the  total 
number  of  engagements,  the  number  of  flank  or  rear  engagements,  and  the  number  of 
non-flank  or  non-rear  engagements.  Data  collection  should  start  at  first  direct  fire 
engagement  with  OPFOR. 

b.  Rationale:  If  the  staff  was  aware  of  the  location  of  the  OPFOR,  it  would  ensure  that 
everyone  in  the  unit  was  aware  of  the  situation  and  direct  the  unit’s  maneuver  to  either 
meet  the  threat  head  on  or  to  a  position  where  the  OPFOR  could  be  attacked  from  the 
flank  or  rear,  increasing  the  probability  of  destroying  the  OPFOR  while  reducing  the  risk 
to  the  unit. 

c.  Output:  Picture  and/or  pie  chart 

3.  Fire  Support  Coordination  -  Comparison  of  OPFOR  kills  due  to  direct  fire  and  indirect  fire 

during  each  mission. 

a.  Operational  definition:  Location  on  the  battlefield  of  each  OPFOR  kill  due  to  indirect 
fire  as  well  as  each  OPFOR  kill  due  to  direct  fire  from  units  controlled  by  the  battalion 
staff.  Additionally,  number  of  OPFOR  kills  due  to  indirect  fire  from  units  controlled  by 
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the  battalion  staff  to  OPFOR  kills  due  to  direet  fire  controlled  by  battalion  subordinate 
units. 

b.  Rationale:  If  the  staff  has  sufficient  situational  awareness  as  well  as  the  ability  to 
synchronize  the  fire  support  assets  under  its  control,  then  it  could  inflict  significant 
damage  to  the  OPFOR  before  the  OPFOR  can  close  to  within  direct  fire  range.  The 
farther  away  from  the  unit  the  OPFOR  can  be  destroyed,  the  less  risk  there  would  be  to 
the  experimental  unit. 

c.  Output:  Picture  and/or  bar  graph 

4.  Air  Defense  Coverage  -  Depiction  of  air  defense  system  range  templates  overlayed  on  the 

location  of  the  unit's  critical  assets  taken  at  a  specified  time. 

a.  Operational  definition:  Location  of  air  defense  systems  on  the  battlefield,  their  range 
templates,  and  the  critical  assets  they  are  covering  at  critical  points  during  each  mission 
(start  of  exercise,  first  indirect  fire  engagement  with  OPFOR,  first  direct  fire  engagement 
with  OPFOR,  first  friendly  casualty;  friendly  losses  exceed  30%,  and  last  engagement 
during  mission). 

b.  Rationale:  Location  of  air  defense  systems  in  relation  to  critical  assets  may  indicate  if  the 
staff  is  monitoring  the  location  of  subordinate  units  and  directing  changes  in  air  defense 
coverage  based  on  the  commander's  priorities. 

c.  Output:  Picture 

5.  Fratricide/Collateral  Damage  -  Depiction  of  the  location,  unit(s)  involved,  and  results  of 

fratricide  and  collateral  damage  during  each  mission. 

a.  Operational  definition:  Each  instance  of  attack  on  BLUFOR  vehicles  and/or  personnel 
by  weapon  systems  under  the  unit’s  control  during  a  mission.  Data  should  reflect  firing 
unit  ID,  echelon  of  controlling  headquarters,  type  of  weapon,  time  of  engagement, 
distance  between  the  shooter  and  the  victim,  and  damage  to  the  targeted  BLUFOR  unit. 

In  addition,  report  each  instance  of  attack  on  non-combatant  or  civilian  vehicles  and/or 
personnel  by  weapon  systems  under  the  unit’s  control  during  a  mission.  Data  should 
reflect  firing  unit  ID,  echelon  of  controlling  headquarters,  type  of  weapon,  time  of 
engagement,  and  damage  to  the  targeted  non-combatant  vehicle  and/or  personnel.  When 
presented  in  picture  format,  the  data  should  also  depict  trace  lines  between  the  shooter 
and  victim. 

b.  Rationale:  If  the  staff  is  keeping  the  rest  of  the  unit  apprised  of  non-combatant  locations 
and  other  friendly  forces  locations  and  activities,  there  should  be  no  instances  of 
fratricide  and  very  few  instances  of  collateral  damage  caused  by  units  equipped  with 
advanced  systems  like  the  SC"*  that  provide  users  with  updated  situational  information 
every  60  seconds  or  less.  Fratricide  and/or  collateral  damage  may  indicate  problems  with 
situational  awareness  within  a  unit.  The  type  of  ammunition  employed  and  the  range 
may  indicate  if  the  unit  used  area-type  or  “dumb”  ammunition  against  targets  that  should 
have  been  engaged  with  point-type  or  “smart”  ammunition. 

c.  Output:  Picture  and/or  table 

6.  Common  Picture  -  Difference  between  electronic  map  displays  at  specified  time  periods. 

a.  Operational  definition:  Comparison  of  the  battlefield  with  BLUFOR  and  detected 

OPFOR  at  critical  points  during  each  mission  (start  of  exercise,  first  indirect  fire 
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engagement  with  OPFOR,  first  direct  fire  engagement  with  OPFOR,  first  friendly 
casualty,  friendly  losses  exceed  30%,  and  last  engagement  during  mission)  and  30 
minutes  after  each  critical  point.  The  pictures  will  indicate  which  previously  detected 
OPFOR  vehicles  disappeared  and  which  ones  were  newly  detected  between  the  two 
times.  Comparisons  will  be  made  among  various  duty  positions  to  determine  the 
commonality  of  displays  among  the  staff 

b.  Rationale:  Data  will  identify  if  the  staff  is  sharing  a  common  picture  of  the  battlefield, 
which  may  be  one  indicator  of  shared  situational  awareness. 

c.  Output:  Picture  series 


Communication 

1 .  Overlay  Use  -  Depiction  of  what  graphics  each  command  post  had  on  its  decision  map  at  a 

specified  time. 

a.  Operational  definition:  Number  of  staff  members  that  are  showing,  on  their  PVD,  the 
same  operations  overlay  file  that  the  battalion  commander  is  showing  on  his  PVD  at 
critical  points  during  each  mission  (start  of  exercise,  first  indirect  fire  engagement  with 
OPFOR,  first  direct  fire  engagement  with  OPFOR,  first  friendly  casualty,  friendly  losses 
exceed  30%,  and  last  engagement  during  mission).  These  data  should  be  reported  as  a 
percentage  by  dividing  the  number  of  staff  members  showing  the  same  overlay  file  as  the 
commander  by  the  total  number  of  staff  members.  If  possible,  report  by  duty  position 
and  operations  overlay  file  name,  those  staff  members  who  are  not  showing  the  same  file 
as  the  commander  at  critical  points  in  time. 

b.  Rationale:  The  data  may  indicate  whether  the  commander  and  his  staff  are  using  the 
same  graphic  control  measures  to  monitor  and  control  subordinate  units.  If  they  are  not, 
then  there  is  a  significant  potential  for  miscommunication  and  a  breakdown  of  situational 
awareness  within  the  unit.  Measures  the  behaviors  related  to  synchronizing  actions, 
passing  information  in  timely  and  efficient  manner,  and  facilitating  performance  of  other 
team  members. 

c.  Output:  Table 


Coordination 

1 .  Damage  to  BLUFOR/OPFOR  Systems  -  Number  and  type  of  BLUFOR/OPFOR  systems  out 

of  action  and  what  damaged  or  destroyed  them  during  each  mission. 

a.  Operational  definition:  Number  of  BLUFOR/OPFOR  tanks,  artillery,  and  air  defense 
system  kills  by  various  categories  of  weapons  for  each  mission. 

b.  Rationale:  More  indirect  fire  kills  than  direct  fire  kills  may  indicate  that  the  staff  has 
effectively  coordinated  fires.  Also,  the  distribution  of  losses  in  the  unit  due  to  indirect 
fire  and  indirect  fire  from  the  OPFOR  may  indicate  whether  the  staff  was  successful 
coordinating  types  of  fire.  Indirect  weapon  systems,  due  to  their  longer  ranges,  are 
habitually  employed  to  attack  the  other  side’s  indirect  fire  weapon  systems. 

c.  Output:  Bar  graph 
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2.  Degradation  of  Forces  -  Depiction  of  the  relative  combat  power  of  maneuver  forces  over 

time  for  both  OPFOR  and  BLUFOR  during  each  mission. 

a.  Operational  definition:  Cumulative  rate  of  OPFOR  destruction  in  20-minute  intervals 
from  the  first  engagement  until  the  last  OPFOR  kill  for  each  mission.  In  addition, 
cumulative  rate  of  BLUFOR  destruction  in  20-minute  intervals  from  the  first  engagement 
until  the  last  BLUFOR  kill  for  each  mission. 

b.  Rationale:  Rapid  destruction  of  the  OPFOR  reduces  the  risk  of  losses  to  the  friendly  unit. 
The  rate  of  loss  also  may  indicate  battle  tempo  during  the  mission  and  whether  the  staff  is 
coordinating  the  efforts  of  the  unit  to  inflict  the  maximum  number  of  losses  in  the 
shortest  period  of  time. 

c.  Output:  Line  graph 

3 .  Counterreconnaissance  Effectiveness  -  Effectiveness  of  the  BLUFOR  counterreconnaissance 

effort  during  each  mission. 

a.  Operational  definition:  Location  where  each  OPFOR  reconnaissance  asset  was  detected 
and  the  location  where  each  detected  OPFOR  reconnaissance  asset  was  killed 
(catastrophic,  firepower,  or  mobility). 

b.  Rationale:  May  indicate  if  the  staff  was  coordinating  fires  effectively  to  negate  the 
OPFOR  ground  reconnaissance  effort.  If  the  OPFOR  cannot  detect  the  unit,  the  unit 
reduces  its  vulnerability.  OPFOR  ground  reconnaissance  vehicles  are  high  priority, 

HPTs  that,  doctrinally,  should  be  among  the  first  targets  attacked  after  they  have  been 
detected. 

c.  Output:  Picture 

4.  Artillery  and  Counterfire  Radar  Coverage  -  Depiction  of  BLUFOR  artillery  templates  (by 

unit  or  type)  overlaid  on  the  location  of  the  unit’s  subordinate  elements  at  a  specified  time. 

a.  Operational  definition:  Location  of  artillery  systems  on  the  battlefield  and  their  range 
templates  at  critical  points  during  each  mission  (start  of  exercise,  first  indirect  fire 
engagement  with  OPFOR,  first  direct  fire  engagement  with  OPFOR,  first  friendly 
casualty,  friendly  losses  exceed  30%,  and  last  engagement  during  mission). 

b.  Rationale:  May  indicate  whether  the  staff  is  effectively  coordinating  the  indirect  fire 
assets  it  directly  controls  to  support  both  its  subordinate  units  and  the  commander’s  intent 
and  concept  of  the  operation.  Typically,  indirect  fire  weapon  systems,  while 
geographically  dispersed,  are  positioned  where  they  can  mass  timely  fires  at  decisive 
points  in  the  operation  and  attack  targets  of  opportunity  while  reducing  their  exposure  to 
OPFOR  counterartillery  fires. 

c.  Output:  Picture 

5.  Kill  Distance  -  Distance  between  shooter  and  target. 

a.  Operational  definition:  Distance  between  OPFOR/BLUFOR  weapon  system  killed  and 
BLUFOR/OPFOR  system  that  killed  it.  Categorize  data  by  indirect  weapon  and  direct 
fire  systems  separately.  Scatterplot  with  a  trend  line  will  reveal  how  far  the  killed 
systems  were  from  the  system  that  killed  them. 

b.  Rationale:  If  the  staff  was  effectively  coordinating  the  fires  of  the  weapons  it  directly 
controlled,  then  the  majority  of  the  kills  they  obtained  should  be  nearer  the  maximum 
effective  range  of  the  weapon  rather  than  the  minimum  effective  range  of  the  weapon. 


C-5 


c.  Output:  Line  graph  and/or  table 


6.  Sensor  -  Shooter  Time  Lag  -  Time  between  first  detection  of  OPFOR  HVT/HPT  by 

BLUFOR  and  when  the  OPFOR  HVT/HPT  was  killed  during  each  mission. 

a.  Operational  definition:  Amount  of  time  between  first  detection  of  an  OPFOR  HVT/HPT 
and  when  it  was  killed.  Also,  location  where  they  were  first  detected  connected  to  the 
location  they  were  killed.  HVT/HPT  targets  will  be  identified  before  the  start  of  the 
exercise.  Normally,  they  might  be  nodes,  air  defense  artillery  (ADA)  systems, 
artillery  systems,  tanks,  and  key  logistic  systems. 

b.  Rationale:  The  shorter  the  time  interval  between  detection  and  kill,  the  better  the  staff 
may  be  in  coordinating  fires. 

c.  Output:  Picture  and/or  bar  graph 


Decision-Making 

1 .  Sensor  Coverage  -  OPFOR  vehicles  detected  as  well  as  those  not  detected  by  BLUFOR 

sensors  during  each  mission. 

a.  Operational  definition:  Location  of  OPFOR  vehicles  on  the  battlefield  that  are  first 
detected  by  squadron  assets.  Also  include  location  of  OPFOR  vehicles  on  the  battlefield 
that  are  undetected. 

b.  Rationale:  If  the  staff  is  not  properly  deploying  and  monitoring  performance  of  their 
sensors,  then  the  information  that  is  being  displayed  to  the  commander  will  be 
incomplete,  which  will  prevent  him  from  making  a  decision  using  all  information  that 
could  be  made  available  to  him. 

c.  Output:  Picture 

2.  Multiple  Fire  Engagements  -  Number  of  OPFOR  vehicles  which  were  engaged  multiple 

times  during  each  mission. 

a.  Operational  definition:  For  each  OPFOR  target  killed,  calculate  the  total  number  of 
additional  engagements  against  it  by  BLUFOR  weapon  systems  after  it  was  killed.  Data 
collected  should  indicate  the  firing  unit,  echelon  of  its  controlling  headquarters,  the  type 
of  weapon  used,  the  engagement  time,  and  should  indieate  at  which  time  the  OPFOR 
target  was  killed  (firepower,  mobility,  and/or  catastrophic).  For  each  OPFOR  target 
engaged  multiple  times,  show  its  location  on  the  battlefield  and  the  location  of  the  units 
that  continued  to  fire  on  it  after  it  was  killed. 

b.  Rationale:  The  data  on  multiple  engagements  of  an  OPFOR  target  may  indicate  whether 
the  situational  awareness  provided  by  the  SC''  system  and  an  effective  unit  fire 
coordination  and  distribution  system  reduced  or  prevented  the  needless  expenditure  of 
ammunition  against  targets  already  destroyed. 

c.  Output:  Picture  and/or  table 

3.  Maneuver  Battle  Sets  -  Disposition  of  OPFOR  and  BLUFOR  at  a  specified  time  period. 

a.  Operational  definition:  Location  of  OPFOR  company  command  vehicles  and  battalion 
subordinate  maneuver  platoon  leader  vehicles  at  critical  points  during  each  mission  (start 
of  exercise,  first  indirect  fire  engagement  with  OPFOR,  first  direct  fire  engagement  with 
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OPFOR,  first  friendly  casualty,  friendly  losses  exceed  30%,  and  last  engagement  during 
mission). 

b.  Rationale:  May  indicate  whether  the  staff  estimates  and  other  information  staff  members 
provided  to  the  commander  allowed  him  to  make  the  right  decision  on  positioning  his 
maneuver  forces  prior  to  the  start  of  the  trial. 

c.  Output:  Picture 

4.  CSS  Available  Over  Time  -  Availability  of  ammunition  and  fuel  during  each  mission. 

a.  Operational  definition:  Percent  of  critical  ammunition  types  (to  be  determined  prior  to 
start  of  mission)  for  20-minute  intervals  for  each  mission,  using  60%  as  a  base  amount  at 
any  given  time. 

b.  Rationale:  May  indicate  if  the  staff  made  the  right  decision  on  positioning  ammunition 
and  fuel  prior  to  start  of  mission. 

c.  Output:  Line  graph 
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Appendix  D 

Sample  Automated  Measure  Macro 


This  appendix  contains  the  Microsoft®  Excel  Macro,  modified  by  Microsoft® 
VisualBasic®,  that  is  used  to  create  the  bar  graph  for  the  Sensor-Shooter  measure. 


Sub  MakeSENSORSHOOTERGraph 0 

I 

'  MakeSENSORSHOOTERGraph  Macro 

'  Macro  recorded  4/18/2001  by  holdenb 

» 

MakeSENSORSHOOTERGraph! 

MakeSENSORSHOOTERGraphG 

MakeSENSORSHOOTERGraphS 

I 

Dim  myFileName  As  String,  myChart  As  String,  myPath  As  String, 
myTime  As  String,  myTitle  As  String,  ThisFile  As  String 
myPath  =  "C:\FCC2\Results\" 
myFileName  =  Format (Now () ,  "mmddyy") 
myTime  =  Application . InputBox ( "Enter  Time") 
myTitle  =  ActiveChart . ChartTitle . Characters . Text 
MsgBox  "Saving  Chart  as"  &  myPath  &  myTitle  &  myFileName  _ 

&  "  "  &  myTime,  vbinformation,  "Save  as:" 

ThisFile  =  myPath  &  myTitle  &  myFileName  &  &  myTime 

Sheets ( "SensorShooterTimeLagGraph" ) .Select 
Sheets ("SensorShooterTimeLagGraph") .Move 
ActiveWorkbook . SaveAs  (ThisFile) 

ActiveWindow . Close 

Application . DisplayAlerts  =  True 

Windows ("Sensor_Shooter_Time_Lag_Graph.xls") .Activate 
ActiveWindow .Close 

Application . DisplayAlerts  =  False 
Windows ( "FCC2Macros . xls" ) .Activate 
End  Sub 

Sub  MakeSENSORSHOOTERGraphl 0 

I 

'  Macro  recorded  4/13/2001  by  holdenb 
'  Macro  revised  5/24/2001  by  holdenb 
'  Activates  data  source  file 
Workbooks . Open  _ 

"C : \FCC2\Common\Sensor_Shooter_Time_Lag_Graph . xls" 

Dim  Count  As  Integer 

Set  myRange  =  ActiveSheet . UsedRange 

Set  FTime  =  myRange . Columns ( "A" ) 
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'Routine  begins  to  convert  times  into  minutes 


Rows ("1:1") .Select 

Selection. Delete  Shift:=xlUp 

Columns ( "D: D" ) .Select 

Selection . NumberFormat  =  "General" 

Range ("El") .Select 


For  Each  R  In  FTime. Cells 
R.Offset(0,  4). Select 
ActiveCell . FormulaRlCl  = 
R.Offset(0,  5). Select 
Act iveCell . FormulaRlCl  = 
R.0ffset(0,  6). Select 
ActiveCell . FormulaRlCl  = 
R.Offset(0,  7). Select 
ActiveCell . FormulaRlCl  = 
If  R. Value  =  ""  Then  End 


"=RC[-4]-RC[-l]" 
"=HOUR(RC [-1] ) " 
"=MINUTE (RC [-2] ) " 
"=(RC[-2] *60)+RC[-l] " 


End  If 


Next  R 
End  Sub 

Sub  MakeSENS0RSH00TERGraph2 0 


'  MakeSENS0RSH00TERGraph2  Macro 
'  Macro  revised  5/24/2001  by  holdenb 

I 

Windows ("Sensor_Shooter_Time_Lag_Graph.xls") .Activate 
Dim  Count  As  Integer 

Set  myRangel  =  ActiveSheet . UsedRange 
Set  FTimel  =  myRangel . Columns ( "A" ) 

Worksheets ( "Sensor_Shooter_Time_Lag_Graph" ) .  _ 

Cells (1,  10) .Value  =  "Time" 

Worksheets ( "Sensor_Shooter_Time_Lag_Graph" ) ._ 

Cells (2,  10) .Value  =  "<30" 

Worksheets ( "Sensor_Shooter_Time_Lag_Graph" ) ._ 

Cells (3,  10). Value  =  "30-59" 

Worksheets ( "Sensor_Shooter_Time_Lag_Graph" ) ._ 

Cells (4,  10) .Value  =  "60-89" 

Worksheets ( "Sensor_Shooter_Time_Lag_Graph" ) ._ 

Cells (5,  10) .Value  =  "90-119" 

Worksheets ( "Sensor_Shooter_Time_Lag_Graph" ) ._ 

Cells (6,  10) .Value  =  "120-149" 

Worksheets ( "Sensor_Shooter_Time_Lag_Graph" ) ._ 
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Cells  (7,  10)  .Value  =  "150-179" 

Worksheets ( "Sensor_Shooter_Time_Lag_Graph" ) . 

Cells  (8,  10). Value  =  ">180" 

Worksheets ( "Sensor_Shooter_Time_Lag_Graph" ) . 
Cells(l,  11) .Value  =  "Number" 

Countl  =  0 
Count2  =  0 
Counts  =  0 
Count4  =  0 
Counts  =  0 
Count6  =  0 
Count7  =  0 


ActiveSheet .Cells (2 , 
Act iveSheet . Cells (3, 
ActiveSheet . Cells (4, 
ActiveSheet .Cells (5, 
ActiveSheet . Cells ( 6, 
ActiveSheet . Cells (7, 
ActiveSheet .Cells ( 8 , 


11) .Value 
11) .Value 
11 ) . Value 
11 ) . Value 
11) .Value 
11) .Value 
11 ) . Value 


=  Countl 
=  Counts 
=  Counts 
=  Count4 
=  Counts 
=  CountO 
=  Counts 


For  Each  R  In  FTimel. Cells 

If  R. Offset (0,  7). Value  <  30  Then 
Countl  =  Countl  +  1 

ActiveSheet . Cells (2 ,  11). Value  =  Countl 

Else 

If  R.Offset(0,  7) .Value  >=  30  And  R.Offset(0,  7) 

Then 

Counts  =  Counts  +  1 

ActiveSheet . Cells ( 3 ,  11) .Value  =  Counts 
Else 

If  R.Offset(0,  7) .Value  >=  60  And  R.Offset(0,  7) 

Then 

Counts  =  Counts  +  1 

ActiveSheet . Cells ( 4 ,  11) .Value  =  Counts 

Else 

If  R.Offset(0,  7) .Value  >=  90  And  R.Offset(0,  7) 

Then 

Count4  =  Count4  +  1 

ActiveSheet .Cells (5,  11). Value  =  Count4 
Else 

If  R.Offset(0,  7) .Value  >=  120  And  R.Offset(0,  7 

Then 

Counts  =  Counts  +  1 

ActiveSheet . Cells ( 6,  11). Value  =  Counts 
Else 


<  60 


<  90 


<  120 


)  <  150 
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If  R.Offset(0,  7) .Value  >=  150  And  R.Offset(0,  7)  <  180 

Then 

Count6  =  Counts  +  1 

ActiveSheet .Cells (7,  11). Value  =  Counts 
Else 

If  R. Offset (0,  7) .Value  >=  180  Then 
Count!  =  Count!  +  1 

ActiveSheet .Cells (8,  11). Value  =  Count! 

End  If 
End  If 
End  If 
End  If 
End  If 
End  If 
End  If 
Next  R 

End  Sub 

Sub  MakeSENSORSHOOTERGraphS 0 

'  MakeSENSORSHOOTERGraphl  Macro 
'  Macro  recorded  4/13/2001  by  holdenb 
'  Macro  revised  5/24/2001  by  holdenb 

'Creates  Bar  Chart 
Charts . Add 

ActiveChart . ChartType  =  xlColumnClustered 
ActiveChart . SetSourceData 

Source : =Sheets ( "Sensor_Shooter_Time_Lag_Graph" ) . Range ( " J1 : K 
8 " ) ,  PlotBy : =xlColumns 

ActiveChart . Location  Where : =xlLocationAsNewSheet ,  Name:=  _ 
"Sens or Shoo terTimeLagGraph" 

With  ActiveChart 

.HasTitle  =  True 

.ChartTitle. Characters. Text  =  "Sensor  -  Shooter  Time 

Lag" 

. Axes (xlCategory,  xlPrimary) . HasTitle  =  True 
. Axes (xlCategory,  xlPrimary) . AxisTitle . Characters . Text  = 

"Time  (Minutes)  from  Detection  to  Kill" 

. Axes (xlValue,  xlPrimary) . HasTitle  =  True 
. Axes (xlValue,  xlPrimary) .AxisTitle . Characters . Text  = 

"Number" 

End  With 

With  ActiveChart .Axes (xlCategory) 
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. HasMa j orGridlines  =  False 
. HasMinorGridlines  =  False 
End  With 

With  ActiveChart . Axes (xlValue) 

. HasMa j orGridlines  =  True 
.HasMinorGridlines  =  False 
End  With 

ActiveChart . HasLegend  =  False 
ActiveChart . PlotArea . Select 
With  Selection . Border 
.Colorindex  =  16 
.Weight  =  xlThin 
.LineStyle  =  xlContinuous 
End  With 

With  Selection . Interior 
.Colorindex  =  2 
. PatternColorIndex  =  1 
. Pattern  =  xlSolid 
End  With 

ActiveChart .Axes (xlValue) . AxisTitle . Select 
With  Selection 

. HorizontalAlignment  =  xlCenter 
. VerticalAlignment  =  xlCenter 
.Orientation  =  xlVertical 
End  With 

ActiveChart . ChartTitle . Select 
Selection .AutoScaleFont  =  True 
With  Selection . Font 
.Name  =  "Arial" 

.FontStyle  =  "Bold" 

.Size  =  14 

.Strikethrough  =  False 
.Superscript  =  False 
.Subscript  =  False 
.OutlineFont  =  False 
.Shadow  =  False 

.Underline  =  xlUnderlineStyleNone 
.Colorindex  =  xlAutomatic 
.Background  =  xlAutomatic 
End  With 

ActiveChart . SeriesCol lection ( 1 ) .Select 
ActiveChart . SeriesCollection (1) . ApplyDataLabels 
Type : =xlDataLabelsShowValue, 

AutoText : =True ,  LegendKey : =False 
ActiveChart . SeriesCollection (1) .ApplyDataLabels 
Type : =xlDataLabelsShowNone, 

AutoText : =True,  LegendKey : =False 
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ActiveChart . PlotArea . Select 
ActiveChart . ChartArea . Select 
ActiveChart . Deselect 
End  Sub 
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